# On Redundancy and Diversity in Cell-based Neural Architecture Search

arXiv: https://arxiv.org/abs/2203.08887 · HTML: https://mlsanity.com/paper/2203.08887/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 50 of its 65 references that are in the map.
- First posted on arXiv 16 Mar 2022.

## Builds on (most cited first, 30 of 50)

- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016)
- [Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour](https://mlsanity.com/paper/1706.02677/) (2017)
- [Learning Transferable Architectures for Scalable Image Recognition](https://mlsanity.com/paper/1707.07012/) (2017)
- [Regularized Evolution for Image Classifier Architecture Search](https://mlsanity.com/paper/1802.01548/) (2018)
- [Efficient Neural Architecture Search via Parameter Sharing](https://mlsanity.com/paper/1802.03268/) (2018)
- [ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware](https://mlsanity.com/paper/1812.00332/) (2018)
- [FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search](https://mlsanity.com/paper/1812.03443/) (2018)
- [NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search](https://mlsanity.com/paper/2001.00326/) (2020)
- [All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously](https://mlsanity.com/paper/1801.01489/) (2018)
- [NAS-Bench-101: Towards Reproducible Neural Architecture Search](https://mlsanity.com/paper/1902.09635/) (2019)
- [SNAS: Stochastic Neural Architecture Search](https://mlsanity.com/paper/1812.09926/) (2018)
- [PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Search](https://mlsanity.com/paper/1907.05737/) (2019)
- [Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluation](https://mlsanity.com/paper/1904.12760/) (2019)
- [BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Search](https://mlsanity.com/paper/1910.11858/) (2019)
- [Understanding and Robustifying Differentiable Architecture Search](https://mlsanity.com/paper/1909.09656/) (2019)
- [Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective](https://mlsanity.com/paper/2102.11535/) (2021)
- [AutoML-Zero: Evolving Machine Learning Algorithms From Scratch](https://mlsanity.com/paper/2003.03384/) (2020)
- [BRP-NAS: Prediction-based NAS using GCNs](https://mlsanity.com/paper/2007.08668/) (2020)
- [Exploring Randomly Wired Neural Networks for Image Recognition](https://mlsanity.com/paper/1904.01569/) (2019)
- [Rethinking Architecture Selection in Differentiable NAS](https://mlsanity.com/paper/2108.04392/) (2021)
- [DARTS-: Robustly Stepping out of Performance Collapse Without Indicators](https://mlsanity.com/paper/2009.01027/) (2020)
- [DrNAS: Dirichlet Neural Architecture Search](https://mlsanity.com/paper/2006.10355/) (2020)
- [NAS evaluation is frustratingly hard](https://mlsanity.com/paper/1912.12522/) (2019)
- [Graph Structure of Neural Networks](https://mlsanity.com/paper/2007.06559/) (2020)
- [Does Unsupervised Architecture Representation Learning Help Neural Architecture Search?](https://mlsanity.com/paper/2006.06936/) (2020)
- [TransNAS-Bench-101: Improving Transferability and Generalizability of Cross-Task Neural Architecture Search](https://mlsanity.com/paper/2105.11871/) (2021)

## Built on by (most cited first, 18 of 18)

- [Neural Architecture Search: Insights from 1000 Papers](https://mlsanity.com/paper/2301.08727/) (2023)
- [Multi-Agent Design: Optimizing Agents with Better Prompts and Topologies](https://mlsanity.com/paper/2502.02533/) (2025)
- [AutoPEFT: Automatic Configuration Search for Parameter-Efficient Fine-Tuning](https://mlsanity.com/paper/2301.12132/) (2023)
- [NAS-Bench-Graph: Benchmarking Graph Neural Architecture Search](https://mlsanity.com/paper/2206.09166/) (2022)
- [Revisiting Neural Networks for Continual Learning: An Architectural Perspective](https://mlsanity.com/paper/2404.14829/) (2024)
- [PA&DA: Jointly Sampling PAth and DAta for Consistent NAS](https://mlsanity.com/paper/2302.14772/) (2023)
- [On the Privacy Risks of Cell-Based NAS Architectures](https://mlsanity.com/paper/2209.01688/) (2022)
- [einspace: Searching for Neural Architectures from Fundamental Operations](https://mlsanity.com/paper/2405.20838/) (2024)
- [Visual Analysis of Neural Architecture Spaces for Summarizing Design Principles](https://mlsanity.com/paper/2208.09665/) (2022)
- [Are Neural Architecture Search Benchmarks Well Designed? A Deeper Look Into Operation Importance](https://mlsanity.com/paper/2303.16938/) (2023)
- [Optimizing Time Series Forecasting Architectures: A Hierarchical Neural Architecture Search Approach](https://mlsanity.com/paper/2406.05088/) (2024)
- [MRF-UNets: Searching UNet with Markov Random Fields](https://mlsanity.com/paper/2207.06168/) (2022)
- [Guided Evolutionary Neural Architecture Search With Efficient Performance Estimation](https://mlsanity.com/paper/2208.06475/) (2022)
- [CARL: Causality-guided Architecture Representation Learning for an Interpretable Performance Predictor](https://mlsanity.com/paper/2506.04001/) (2025)
- [Achieving Deep Continual Learning via Evolution](https://mlsanity.com/paper/2502.06210/) (2025)
- [Multi-Task Neural Architecture Search Using Architecture Embedding and Transfer Rank](https://mlsanity.com/paper/2504.00772/) (2025)
- [Architecture Before the Formula: Training-Dependent Computational Semantics in Transformers](https://mlsanity.com/paper/2601.11618/) (2026)
- [GraDE: A Graph Diffusion Estimator for Frequent Subgraph Discovery in Neural Architectures](https://mlsanity.com/paper/2602.03257/) (2026)

## Abstract (arXiv)

Searching for the architecture cells is a dominant paradigm in NAS. However, little attention has been devoted to the analysis of the cell-based search spaces even though it is highly important for the continual development of NAS. In this work, we conduct an empirical post-hoc analysis of architectures from the popular cell-based search spaces and find that the existing search spaces contain a high degree of redundancy: the architecture performance is minimally sensitive to changes at large parts of the cells, and universally adopted designs, like the explicit search for a reduction cell, significantly increase the complexities but have very limited impact on the performance. Across architectures found by a diverse set of search strategies, we consistently find that the parts of the cells that do matter for architecture performance often follow similar and simple patterns. By explicitly constraining cells to include these patterns, randomly sampled architectures can match or even outperform the state of the art. These findings cast doubts into our ability to discover truly novel architectures in the existing cell-based search spaces, and inspire our suggestions for improvement to guide future NAS research. Code is available at https://github.com/xingchenwan/cell-based-NAS-analysis.