介绍

1. we focus on…but… #创新点
2. However, the use of … for the discovery of …, an important element in …, is limited within the literature. #创新点
3. This work focuses on utilizing … to examine the …, which follow a fundamental DenseNet structure, but have no …. #创新点
4. The structures found by the algorithm are examined to shed light on the importance of …, including the discovery of a simple skip-connection removal, which improves … performance on ….
5. CLASSIFICATION is among the most important machine learning tasks.分类是最重要的机器学习任务之一。 #研究介绍
6. 2. This work focuses on utilizing evolutionary neural architecture search to examine the search space of networks, which follow a fundamental DenseNet structure, but have no fixed skip connections. #研究介绍
7. Within the CNN design, the recent work in neural architecture search (NAS) has demonstrated that automatically designed networks can match, and even surpass, the performance of CNNs designed by human experts, and has led to the discovery of novel state-of-the-art CNN architectures with modules, which are unlikely to be developed by human experts .在CNN设计中，最近在神经架构搜索（NAS）方面的工作表明，自动设计的网络可以匹配甚至超越人类专家设计的CNN的性能，并导致发现了具有模块的新颖的最先进的CNN架构，而人类专家不太可能开发这些架构。 #肯定研究成果
8. However, the most competitive NAS algorithms utilize predefined, simple, skip-connection structures, despite the apparent importance of skip connections and the use of distinct, competing styles of skip connection appearing in state-of-the-art works 然而，最具竞争力的NAS算法利用了预定义的、简单的跳过连接结构，尽管跳过连接显然很重要，而且在最先进的作品中出现了不同的、竞争性的跳过连接样式 #研究介绍
9. With regards to the breadth of scope of NAS algorithms, minimal research has been conducted with a focus on the automatic discovery of skip-connection structure in CNNs, and no published work has demonstrated an algorithm, which can successfully optimize skip-connection structure when the possible number of skip connections is large.关于 NAS 算法范围的广度，以 CNN 中跳跃连接结构的自动发现为重点的研究很少，也没有发表的作品展示了一种算法，当 可能的跳过连接数很大。
10. In practical applications, the discovery of novel skipconnection structures in deep networks provides the opportunity to optimize networks to be both smaller and more accurate for a given task, and do so in a way that does not conflict with existing methods which optimize the layers used in networks. This is of particular importance when considering the use of artificial neural networks in computation limited devices, such as mobile phones and autonomous vehicles .在实际应用中，深度网络中新颖的 skipconnection 结构的发现为优化网络提供了机会，使其针对给定任务变得更小、更准确，并且这样做的方式与现有的优化层的方法不冲突 网络。 当考虑在移动电话和自动驾驶汽车等计算受限设备中使用人工神经网络时，这一点尤为重要 [13]。 #skipconnection
11. To accomplish this idea, we propose the PONAS method to combine advantages of progressive NAS and oneshot methods. 为了实现这个想法
12. The two main approaches to neural architecture search are reinforcement learning and evolutionary algorithms; we focus on the evolutionary algorithm approach in this paper. 两种方法，我们关注遗传算法搜索 #引出ENAS
13. Neural architecture search (NAS) is a family of algorithms and techniques for designing neural networks. The two most popular approaches are reinforcement learning and evolutionary techniques. We use an evolutionary search algorithm in this work because evolutionary algorithms are simpler to implement and understand in addition to typically requiring less computational resources when compared to their reinforcement learning based counterparts. The main challenge the evolutionary algorithm needs to overcome is that of exploration versus exploitation. We need to make sure we give every architecture a fair evaluation in the sense that it has trained long enough such that the evaluation phase, where it is tested on validation data, accurately assesses its effectiveness on the task at hand. On the other hand, given a finite amount of computational resources, we want to maximize the number of explored architectures. These two aspects of evolutionary neural architecture search are at odds with each other — longer training time consumes computational resources, leaving less resources for the remaining search神 经 体 系 结 构搜 索 ( N eu ral ar c hitectu re s e ar ch, N A S)是 一 组用 于 设 计 神 经 网络 的 算 法 和 技 术 。 最 流 行 的 两 种方 法 是 强 化学 习 和 进 化 技 术。 我 们 在 这 项 工 作 中 使 用 进 化 搜索 算 法 ， 因为 与 基 于 强 化 学习 的 同 类 算 法 相 比 ， 进 化 算 法 实现 和 理 解 起来 更 简 单 ， 而 且通 常 需 要 更 少 的 计 算 资 源 。 进 化算 法 需 要 克服 的 主 要 挑 战 是探 索 与 利 用 的 对 比 。 我 们 需 要 确保 我 们 给 每个 架 构 一 个 公 平的 评 估 ， 从 这 个 意 义 上 说 ， 它 已经 训 练 了 足够 长 的 时 间 ， 以便 在 评 估 阶 段 ， 在 验 证 数 据 上 测试 它 ， 准 确地 评 估 它 在 手 头任 务 上 的 有 效 性 。 另 一 方 面 ， 给定 有 限 的 计算 资 源 ， 我 们 希望 最 大 化 探 索 的 架 构 的 数 量 。 进化 神 经 架 构搜 索 的 这 两 个 方面 是 相 互 矛 盾 的 — — 训 练 时 间 越长 消 耗 计 算资源，留给 剩余 搜 索 的 资 源 就 越 少 #引出ENAS
14. Although there has been extensive work exploring differentaspects of variational autoencoders [7], [17]–[19] , to the bestof our knowledge there is no prior work exploring evolutionaryneural architecture search techniques to designing variationalautoencoders. Previous work most similar to this topic isthat of [20], [21] explores the use of evolutionary neuralarchitecture search techniques to autoencoders. Aside fromour focus on variational autoencoders rather than traditionalautoencoders, our work differs in that it uses an evolutionaryalgorithm that always generates valid network architecturesand does not require the encoder and decoder networks to bethe same architecture.Much work has been done on the domain of neural architecture search, mostly applied to classification. Work such as [8],[11], [22] focuses on reinforcement learning based approachedto neural architecture search. More related to our work is that of [9], [23], [24], which focuses on evolutionary based applied to variational autoencoders.other areas of application, such as neural architecture searchtion with our work, many of these techniques are applicable tolearning generative models. Despite the difference in applicaral architecture focuses on image classification, rather thanlearning and evolutionary algorithm-based approaches to neuapproaches. A majority of the prior work in reinforcement尽管已经有大量工作探索变分自动编码器的不同方面 [7]、[17]-[19]，但据我们所知，之前没有探索进化神经架构搜索技术来设计变分自动编码器的工作。 与该主题最相似的先前工作是 [20]、[21] 的工作探索了进化神经结构搜索技术在自动编码器中的应用。 除了我们关注变分自动编码器而不是传统自动编码器之外，我们的工作不同之处在于它使用一种进化算法，该算法始终生成有效的网络架构并且不需要编码器和解码器网络是相同的架构。在神经架构搜索领域已经做了很多工作 ，主要应用于分类。 [8]、[11]、[22] 等工作侧重于基于强化学习的神经架构搜索方法。 与我们的工作更相关的是 [9]、[23]、[24] 的工作，它侧重于基于进化的应用于变分自动编码器。其他应用领域，例如神经架构搜索与我们的工作，许多这些技术是适用的 学习生成模型。 尽管应用架构有所不同，但侧重于图像分类，而不是基于学习和进化算法的新方法。 大部分先前的加固工作 #引出ENAS
15. In this paper, the research topics of NAS are organized inthe way illustrated in Fig. 1. From a macroscopic perspective,NAS components can be categorized as the search space definition, search strategy, and performance evaluation criterion.。从宏观的角度来看， NAS 组件可以分为搜索空间定义、搜索策略和性能评估标准 #引出ENAS

经验

18. 我们提出的编码方法中，我们避免使用全连接层。这与相关 工作[15]、[16]中采用的方法一致；全连接层由于其密集连接而容易过度拟合[19]。2019-A_Graph-Based_Encoding_for_Evolutionary_Convolutional_Neural_Network_Architecture_Design-1 #经验

总结性

1. 有基于细胞的搜索，有基于个体的搜索
   基于细胞Particle_Swarm_optimisation_for_Evolving_Deep_Neural_Networks_for_Image_Classification_by_Evolving_and_Stacking_Transferable_Blocks

图介绍

研究唯一性

1. This is the first work that employs NAS for MPNNs and establishes the first graph-related search space to incorporate edge features with skip connections.这是第一个将 NAS 用于 MPNN 的工作，并建立了第一个与图相关的搜索空间，以将边缘特征与跳跃连接结合起来。

NAS介绍

1. NAS can be viewed as a bilevel optimization problem and mathematically formulated as follows: train valid #NAS
2. RL-based NAS methods adopt a controller to sample a new candidate network to be trained and its performance is used as the reward score. Then, the reward score can be adopted to update the controller for sampling a better candidate network in the next iteration. #NAS-强化学习
3. 5. how these networks are represented within the evolutionary NAS algorithm. 这些网络如何在演化NAS中表示 #NAS
4. Neural architecture search (NAS) has been designed to automatically search for the best network architecture for a given dataset. Specifically, it uses a search method (e.g., reinforcement learning, evolutionary algorithm, or stochastic gradient descent) to explore the user-defined search space and chooses the best architecture based on the performance of the generated model (e.g., validation accuracy) on a given task. The search space contains all possible architectures. The architectures discovered by NAS have been shown to outperform manually designed architectures in various tasks such as image classification [13], image segmentation [14], natural language processing [15], and time series prediction 神经架构搜索 (NAS) 旨在自动搜索给定数据集的最佳网络架构。 具体来说，它使用一种搜索方法（例如，强化学习、进化算法或随机梯度下降）来探索用户定义的搜索空间，并根据生成模型的性能（例如，验证准确性）选择最佳架构。 给定的任务。 搜索空间包含所有可能的架构。 NAS 发现的架构已被证明在各种任务中优于手动设计的架构，例如图像分类 [13]、图像分割 [14]、自然语言处理 [15] 和时间序列预测
5. Different from the previous neuroevolution techniques [35] that aim to optimize both the weights and architecture of neural networks, EA-based NAS only optimizes the model architecture itself, and the model parameters are trained using conventional gradient descent methods [11]–[14].不同于以往旨在同时优化神经网络权重和架构的神经进化技术[35]，基于EA的NAS仅优化模型架构本身，模型参数使用传统的梯度下降方法[11]-[14]进行训练 #NAS-演化
6. Since EAs are particularly well suited to dealing with multiobjective optimization problems, multiobjective evolutionary NAS has received increased attention. 由于 EA 特别适合处理多目标优化问题，因此多目标进化 NAS 受到了越来越多的关注。 #NAS-多目标演化

NAS必要性

1. The manually designed MPNNs not only require a substantial number of experiments in the design space but also tend to be lowerperforming when applied to a new dataset.手动设计的 MPNN 不仅需要在设计空间中进行大量实验，而且在应用于新数据集时往往表现不佳。 #NAS-必要性

继承必要性

1. These elite individuals directly pass down to the next generation, which preserve their architectures and weights and reduce the retraining cost.这些精英个体直接传给下一代，保留了他们的架构和权重，降低了再培训成本。 #继承-必要性
2. 1. However, the full weight sharing training paradigm in OSNAS may result in strong interference across candidate architectures and mislead the architecture search. #继承-劣势
3. The efficiency of performance estimation of the candidate architectures is therefore greatly enhanced because it avoids training a large number of candidate models from scratch.
   oneshot可以不用反复训练的优势。我的论文基于块的节点继承，大部分研究为了减少评估运算量使用的是整个 #继承-必要性

搜索空间

1. The generality of the chosen search space has a major influence on the quality of results that are even possible.所选搜索空间的普遍性对甚至可能的结果质量有重大影响。 #search-space

   变异

2. The benefit of the proposed mutation operation is to make the one-shot model training less likely to get trapped in local minimums. #mutation

多样性

1. Also, in evolutionary algorithms, higher population diversity contributes to locating global optimal solutions and avoiding early convergence [27]. #多样性-必要

2. 研究稀缺

3. However, the use of neural architecture search for the discovery of skip-connection structures, an important element in modern convolutional neural networks, is limited within the literature. #研究稀缺性-skipconnection
4. To the best of our knowledge, we firstly apply attention-based graph representation learning to fraud detection, which has yielded significant results in identifying fraudulent calls without raising too many false alarms. #研究稀缺性-通用

深度学习（夸赞）

1. However, traditional machine learning techniques are limited in processing data in their raw form and require a lot of human intervention. Deep learning techniques are more suitable than traditional machine learning techniques for various classification tasks, including malware detection.Hence, research focusing on deep learning for malware detection, especially Android malware, has recently received more attention.然而传统的机器学习受限于。。。。深度学习更适合 #夸赞-深度学习
2. 7. In particular, the introduction of skip-connections has allowed for CNNs to achieve previously unattainable results in both handcrafted and automatically designed networks.特别是，跳过连接的引入使CNNs能够在手工和自动设计的网络中实现以前无法实现的结果。 #夸赞-skipconnection
3. The final algorithm finds networks that are more accurate than DenseNets on CIFAR10 and CIFAR100, and have fewer trainable parameters. #夸赞-skipconnection #实验结论

代理（必要性）

1. Since it is impossible to evaluate the performance of all architectures through the training and validation procedure within a limited search cost, some kind of search strategy is always required for NAS.资源有限，需要使用某种策略降低复杂度。
2. However, training a deep neural network requires a large amount of time and computational resources. So the fitness evaluation of an individual is extremely expensive. We attempt to use a classifier as the surrogate model to learn the approximate performance of each network architecture for eliminating bad offspring and retaining good ones.然而，训练深度神经网络需要大量的时间和计算资源。 所以对一个人的适应度评价是极其昂贵的。 我们尝试使用分类器作为替代模型来学习每个网络架构的近似性能，以消除不良后代并保留良好后代。

研究必要性

1. With the surge service requirements from vehicular users, especially for automated driving, providing real-time high-rate wireless connections to fast-moving vehicles is ever demanding. 随着车辆用户对服务需求的激增，尤其是对自动驾驶的需求，为快速移动的车辆提供实时高速无线连接的要求越来越高。

   贡献

2. The major contributions of this paper are four-fold.

信息安全（研究）的必要性

1. Traditional malware detection techniques cannot cope with this problem due to the rapid evolution of complex malware or the emergence of zero-day malware.由于复杂恶意软件的快速发展或零日恶意软件的出现，传统的恶意软件检测技术无法应对此问题。

工作借鉴

1. we adopt the one used in [8], [9], [17], and [31] to be consistent with previous work #工作借鉴
2. Schematic of the NSGANetV1 search space motivated from [8]. #工作借鉴-搜索空间

   研究结果

3. As a result, optimal networks obtained by Evo-OSNAS can achieve more competitive performance than the state-of-the-art networks found by existing NAS algorithms. #夸赞-研究结果

   算法实现

4. Our objective is to … #目标

验证

1. We will verify Conjecture 1 on real-world datasets through our experiments (in Section 4). #实验
2. The effectiveness of the different architectures is judged on both classification accuracy and computational complexity.

NAS评估成本

1. Furthermore, the high computational cost of NAS algorithms has led to many NAS algorithms utilizing performance estimation techniques to reduce computational cost. However, empirical justifications of the performance estimation techniques used are frequently lacking, leaving much NAS research in the dark with respect to the ramifications of these choices and possible alternatives to the chosen techniques [6].此外，NAS算法的高计算成本导致许多NAS算法利用性能估计技术来降低计算成本。 然而，所使用的性能估计技术的经验论据经常缺乏，使得许多 NAS 研究在这些选择的后果和所选技术的可能替代方案方面处于黑暗之中 [6]。

Flops

1. NSGA-Net also proves that FLOPs is a more accurate and reliable parameter of network complexity. #flops

   改进ENAS

2. In order to enhance the exploration ability of EA for NAS, we adopt a one-to-many mutation strategy and a neural predictor to guide the evolution of EA. #改进-ENAS

   E-NAS研究现状

3. 《A Cell-Based Fast Memetic Algorithm for Automated Convolutional Neural Architecture Design》In contrast to RL- and gradient-based algorithms, the ES-based algorithm is a type of nature-inspired populationbased optimization approach, which has strong global search capability and has no requirements, such as convex nature, derivability, and explicit mathematic formulation, on the objective functions. Over the years, quite a number of ES-based algorithms have been proposed for NAS. In particular, one of the earliest ES-based methods is NeuroEvolution of Augmenting Topologies (NEAT) proposed by Stanley and Miikkulainen [17], which optimizes the topology structure and weights of a neural network simultaneously. Next, Real et al. [18] employed simple evolutionary techniques to discover neural network architectures for the CIFAR-10 and CIFAR-100 datasets. Xie and Yuille [19] used a genetic algorithm with an encoding method representing each network structure in a fixed-length binary string to learn deep CNN structures automatically. To further improve the NAS performance, Real et al. modified the tournament selection evolutionary algorithm (EA) by introducing an age property to favor the younger genotypes, which obtained superior performance over handcrafted models [20]. Sun et al. [21] proposed a variable-length encoding technique and a novel fitness evaluation method in the genetic algorithm to explore the architecture space efficiently. More recently, Yang et al. [22] developed an efficient continuous evolutionary approach for searching neural networks, which directly inherits both the SuperNet and the population over generations. Zhang et al. proposed a computationally efficient ES of convolutional networks based on a directed acyclic graph (DAG), which randomly samples and trains parents on each minibatch of the training data. However, despite the attractive global optimization capability of ES, it is noted that ES requires a large number of performance evaluations in its iterative search process, while the training of deep neural networks is itself a computationally expensive task.与基于 RL 和梯度的算法相比，基于 ES 的算法是一种基于自然启发的基于种群的优化方法，具有强大的全局搜索能力，并且没有凸性、可推导性和显式数学公式等要求， 关于目标函数。 多年来，已经为 NAS 提出了相当多的基于 ES 的算法。 特别是，最早的基于 ES 的方法之一是由 Stanley 和 Miikkulainen [17] 提出的增强拓扑结构的神经进化 (NEAT)，它同时优化了神经网络的拓扑结构和权重。 接下来，Real 等人。 [18] 采用简单的进化技术来发现 CIFAR-10 和 CIFAR-100 数据集的神经网络架构。 Xie 和 Yuille [19] 使用一种遗传算法和一种编码方法，以固定长度的二进制字符串表示每个网络结构来自动学习深度 CNN 结构。 为了进一步提高 NAS 性能，Real 等人。 通过引入年龄属性来支持较年轻的基因型，修改了锦标赛选择进化算法 (EA)，该算法获得了优于手工模型的性能 [20]。 孙等。 [21] 在遗传算法中提出了一种可变长度编码技术和一种新的适应度评估方法，以有效地探索体系结构空间。 最近，杨等人。 [22] 开发了一种有效的连续进化方法来搜索神经网络，它直接继承了 SuperNet 和代代相传的种群。 张等。 提出了一种基于有向无环图 (DAG) 的卷积网络的计算高效 ES，它在训练数据的每个小批量上随机采样和训练父母。 然而，尽管 ES 具有吸引人的全局优化能力，但值得注意的是，ES 在其迭代搜索过程中需要进行大量的性能评估，而深度神经网络的训练本身就是一项计算量大的任务。

   章节介绍

4. The pipeline of the EA is shown in Fig. 2(a). #介绍-流程图
5. The encoding method of neural architecture is introduced in Section III-B. #介绍-章节
6. The proposed neural predictors are expounded in Section III-C. #介绍-章节
7. The combination of the EA with the graph-based uncertainty estimation network is outlined in Section III-D1, and the combination of the EA with the graph-based neural predictor is discussed in Section III-D2. Moreover, the random architecture sampling methods are analyzed in Section III-E. #介绍-章节

8. 单词

   结论

9. accordingly, 因此