# Multi-Objective Evolutionary Neural Architecture Search for Recurrent Neural Networks

arXiv: https://arxiv.org/abs/2403.11173 · HTML: https://mlsanity.com/paper/2403.11173/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 59 references that are in the map.
- First posted on arXiv 17 Mar 2024.

## Builds on (most cited first, 30 of 30)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016)
- [Efficient Neural Architecture Search via Parameter Sharing](https://mlsanity.com/paper/1802.03268/) (2018)
- [AutoML: A Survey of the State-of-the-Art](https://mlsanity.com/paper/1908.00709/) (2019)
- [Random Search and Reproducibility for Neural Architecture Search](https://mlsanity.com/paper/1902.07638/) (2019)
- [Regularizing and Optimizing LSTM Language Models](https://mlsanity.com/paper/1708.02182/) (2017)
- [Exploring the Limits of Language Modeling](https://mlsanity.com/paper/1602.02410/) (2016)
- [Visualizing and Understanding Recurrent Networks](https://mlsanity.com/paper/1506.02078/) (2015)
- [Efficient Multi-objective Neural Architecture Search via Lamarckian Evolution](https://mlsanity.com/paper/1804.09081/) (2018)
- [Breaking the Softmax Bottleneck: A High-Rank RNN Language Model](https://mlsanity.com/paper/1711.03953/) (2017)
- [A Survey on Evolutionary Neural Architecture Search](https://mlsanity.com/paper/2008.10937/) (2020)
- [CARS: Continuous Evolution for Efficient Neural Architecture Search](https://mlsanity.com/paper/1909.04977/) (2019)
- [NSGANetV2: Evolutionary Multi-Objective Surrogate-Assisted Neural Architecture Search](https://mlsanity.com/paper/2007.10396/) (2020)
- [A Survey on Neural Architecture Search](https://mlsanity.com/paper/1905.01392/) (2019)
- [Network Morphism](https://mlsanity.com/paper/1603.01670/) (2016)
- [How to Construct Deep Recurrent Neural Networks](https://mlsanity.com/paper/1312.6026/) (2013)
- [NAS-Bench-NLP: Neural Architecture Search Benchmark for Natural Language Processing](https://mlsanity.com/paper/2006.07116/) (2020)
- [Gated Feedback Recurrent Neural Networks](https://mlsanity.com/paper/1502.02367/) (2015)
- [A Study on Encodings for Neural Architecture Search](https://mlsanity.com/paper/2007.04965/) (2020)
- [Multi-Objective Reinforced Evolution in Mobile Neural Architecture Search](https://mlsanity.com/paper/1901.01074/) (2019)
- [On Evaluating the Generalization of LSTM Models in Formal Languages](https://mlsanity.com/paper/1811.01001/) (2018)
- [A Gentle Tutorial of Recurrent Neural Network with Error Backpropagation](https://mlsanity.com/paper/1610.02583/) (2016)
- [Multi-objective Neural Architecture Search via Non-stationary Policy Gradient](https://mlsanity.com/paper/2001.08437/) (2020)
- [Improved Crowding Distance for NSGA-II](https://mlsanity.com/paper/1811.12667/) (2018)
- [Language Modeling through Long Term Memory Network](https://mlsanity.com/paper/1904.08936/) (2019)
- [Multi-Objective Neural Architecture Search Based on Diverse Structures and Adaptive Recommendation](https://mlsanity.com/paper/2007.02749/) (2020)
- [Multi-objective Neural Architecture Search with Almost No Training](https://mlsanity.com/paper/2011.13591/) (2020)

## Built on by (most cited first, 2 of 2)

- [LLM-NAS: LLM-driven Hardware-Aware Neural Architecture Search](https://mlsanity.com/paper/2510.01472/) (2025)
- [SiamNAS: Siamese Surrogate Model for Dominance Relation Prediction in Multi-objective Neural Architecture Search](https://mlsanity.com/paper/2506.02623/) (2025)

## Abstract (arXiv)

Artificial neural network (NN) architecture design is a nontrivial and time-consuming task that often requires a high level of human expertise. Neural architecture search (NAS) serves to automate the design of NN architectures and has proven to be successful in automatically finding NN architectures that outperform those manually designed by human experts. NN architecture performance can be quantified based on multiple objectives, which include model accuracy and some NN architecture complexity objectives, among others. The majority of modern NAS methods that consider multiple objectives for NN architecture performance evaluation are concerned with automated feed forward NN architecture design, which leaves multi-objective automated recurrent neural network (RNN) architecture design unexplored. RNNs are important for modeling sequential datasets, and prominent within the natural language processing domain. It is often the case in real world implementations of machine learning and NNs that a reasonable trade-off is accepted for marginally reduced model accuracy in favour of lower computational resources demanded by the model. This paper proposes a multi-objective evolutionary algorithm-based RNN architecture search method. The proposed method relies on approximate network morphisms for RNN architecture complexity optimisation during evolution. The results show that the proposed method is capable of finding novel RNN architectures with comparable performance to state-of-the-art manually designed RNN architectures, but with reduced computational demand.