# Long-term Reproducibility for Neural Architecture Search

arXiv: https://arxiv.org/abs/2207.04821 · HTML: https://mlsanity.com/paper/2207.04821/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 20 references that are in the map.
- First posted on arXiv 11 Jul 2022.

## Builds on (most cited first, 16 of 16)

- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016)
- [Learning Transferable Architectures for Scalable Image Recognition](https://mlsanity.com/paper/1707.07012/) (2017)
- [Efficient Neural Architecture Search via Parameter Sharing](https://mlsanity.com/paper/1802.03268/) (2018)
- [ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware](https://mlsanity.com/paper/1812.00332/) (2018)
- [Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)](https://mlsanity.com/paper/2003.12206/) (2020)
- [PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Search](https://mlsanity.com/paper/1907.05737/) (2019)
- [Random Search and Reproducibility for Neural Architecture Search](https://mlsanity.com/paper/1902.07638/) (2019)
- [Neural Architecture Search without Training](https://mlsanity.com/paper/2006.04647/) (2020)
- [Efficient Multi-objective Neural Architecture Search via Lamarckian Evolution](https://mlsanity.com/paper/1804.09081/) (2018)
- [Blockwisely Supervised Neural Architecture Search with Knowledge Distillation](https://mlsanity.com/paper/1911.13053/) (2019)
- [DrNAS: Dirichlet Neural Architecture Search](https://mlsanity.com/paper/2006.10355/) (2020)
- [Neural Architecture Transfer](https://mlsanity.com/paper/2005.05859/) (2020)
- [Best Practices for Scientific Research on Neural Architecture Search](https://mlsanity.com/paper/1909.02453/) (2019)
- [Bonsai-Net: One-Shot Neural Architecture Search via Differentiable Pruners](https://mlsanity.com/paper/2006.09264/) (2020)
- [SpiderNet: Hybrid Differentiable-Evolutionary Architecture Search via Train-Free Metrics](https://mlsanity.com/paper/2204.09320/) (2022)

## Built on by (most cited first, 1 of 1)

- [Inaccessible Neural Language Models Could Reinvigorate Linguistic Nativism](https://mlsanity.com/paper/2301.05272/) (2023)

## Abstract (arXiv)

It is a sad reflection of modern academia that code is often ignored after publication -- there is no academic 'kudos' for bug fixes / maintenance. Code is often unavailable or, if available, contains bugs, is incomplete, or relies on out-of-date / unavailable libraries. This has a significant impact on reproducibility and general scientific progress. Neural Architecture Search (NAS) is no exception to this, with some prior work in reproducibility. However, we argue that these do not consider long-term reproducibility issues. We therefore propose a checklist for long-term NAS reproducibility. We evaluate our checklist against common NAS approaches along with proposing how we can retrospectively make these approaches more long-term reproducible.