# NAAS: Neural Accelerator Architecture Search arXiv: https://arxiv.org/abs/2105.13258 ยท HTML: https://mlsanity.com/paper/2105.13258/ ## Key facts - Cited by 24 papers in the map (data through 14 Oct 2026). - Builds on 15 of its 31 references that are in the map. - First posted on arXiv 27 May 2021. ## Builds on (most cited first, 15 of 15) - [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019) - [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016) - [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019) - [Once-for-All: Train One Network and Specialize it for Efficient Deployment](https://mlsanity.com/paper/1908.09791/) (2019) - [ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware](https://mlsanity.com/paper/1812.00332/) (2018) - [HAQ: Hardware-Aware Automated Quantization with Mixed Precision](https://mlsanity.com/paper/1811.08886/) (2018) - [Single Path One-Shot Neural Architecture Search with Uniform Sampling](https://mlsanity.com/paper/1904.00420/) (2019) - [Learning to Optimize Tensor Programs](https://mlsanity.com/paper/1805.08166/) (2018) - [Hardware/Software Co-Exploration of Neural Architectures](https://mlsanity.com/paper/1907.04650/) (2019) - [FPGA/DNN Co-Design: An Efficient Design Methodology for IoT Intelligence on the Edge](https://mlsanity.com/paper/1904.04421/) (2019) - [Co-Exploration of Neural Architectures and Heterogeneous ASIC Accelerator Designs Targeting Multiple Tasks](https://mlsanity.com/paper/2002.04116/) (2020) - [EDD: Efficient Differentiable DNN Architecture and Implementation Co-search for Embedded AI Solutions](https://mlsanity.com/paper/2005.02563/) (2020) - [ConfuciuX: Autonomous Hardware Resource Assignment for DNN Accelerators using Reinforcement Learning](https://mlsanity.com/paper/2009.02010/) (2020) - [SkyNet: a Hardware-Efficient Method for Object Detection and Tracking on Embedded Systems](https://mlsanity.com/paper/1909.09709/) (2019) - [On Neural Architecture Search for Resource-Constrained Hardware Platforms](https://mlsanity.com/paper/1911.00105/) (2019) ## Built on by (most cited first, 24 of 24) - [Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications](https://mlsanity.com/paper/2204.11786/) (2022) - [Mobile Foundation Model as Firmware](https://mlsanity.com/paper/2308.14363/) (2023) - [AccelTran: A Sparsity-Aware Accelerator for Dynamic Inference with Transformers](https://mlsanity.com/paper/2302.14705/) (2023) - [A Survey on Graph Neural Network Acceleration: Algorithms, Systems, and Customized Hardware](https://mlsanity.com/paper/2306.14052/) (2023) - [CATransformers: Carbon Aware Transformers Through Joint Model-Hardware Optimization](https://mlsanity.com/paper/2505.01386/) (2025) - [Special Session: Towards an Agile Design Methodology for Efficient, Reliable, and Secure ML Systems](https://mlsanity.com/paper/2204.09514/) (2022) - [CODEBench: A Neural Architecture and Hardware Accelerator Co-Design Framework](https://mlsanity.com/paper/2212.03965/) (2022) - [DOSA: Differentiable Model-Based One-Loop Search for DNN Accelerators](https://mlsanity.com/paper/2509.10702/) (2025) - [Algorithm and Hardware Co-design for Reconfigurable CNN Accelerator](https://mlsanity.com/paper/2111.12787/) (2021) - [EdgeTran: Co-designing Transformers for Efficient Inference on Mobile Edge Platforms](https://mlsanity.com/paper/2303.13745/) (2023) - [A Semi-Decoupled Approach to Fast and Optimal Hardware-Software Co-Design of Neural Accelerators](https://mlsanity.com/paper/2203.13921/) (2022) - [GANDSE: Generative Adversarial Network based Design Space Exploration for Neural Network Accelerator Design](https://mlsanity.com/paper/2208.00800/) (2022) - [TransCODE: Co-design of Transformers and Accelerators for Efficient Training and Inference](https://mlsanity.com/paper/2303.14882/) (2023) - [ShiftAddAug: Augment Multiplication-Free Tiny Neural Network with Hybrid Computation](https://mlsanity.com/paper/2407.02881/) (2024) - [Combining Neural Architecture Search and Automatic Code Optimization: A Survey](https://mlsanity.com/paper/2408.04116/) (2024) - [NASH: Neural Architecture and Accelerator Search for Multiplication-Reduced Hybrid Models](https://mlsanity.com/paper/2409.04829/) (2024) - [LEGO: Spatial Accelerator Generation and Optimization for Tensor Applications](https://mlsanity.com/paper/2509.12053/) (2025) - [DRAGON (Differentiable Graph Execution) : A suite of Hardware Simulation and Optimization tools for Modern AI/Non-AI Workloads](https://mlsanity.com/paper/2204.06676/) (2022) - [SONAR: Joint Architecture and System Optimization Search](https://mlsanity.com/paper/2208.12218/) (2022) - [Enabling Hard Constraints in Differentiable Neural Network and Accelerator Co-Exploration](https://mlsanity.com/paper/2301.09312/) (2023) - [Exploring Quantization and Mapping Synergy in Hardware-Aware Deep Neural Network Accelerators](https://mlsanity.com/paper/2404.05368/) (2024) - [HERCULES: Hardware-Efficient, Robust, Continual Learning Neural Architecture Search](https://mlsanity.com/paper/2605.04103/) (2026) - [Uncertainty-Aware End-to-End Co-Design of Neural Network Processors: From Training and Mapping to Fabrication](https://mlsanity.com/paper/2606.04850/) (2026) - [NAS-Driven Hardware Accelerator Exploration for Edge AI and Quantization Effects on the Pareto Space](https://mlsanity.com/paper/2608.13293/) (2026) ## Abstract (arXiv) Data-driven, automatic design space exploration of neural accelerator architecture is desirable for specialization and productivity. Previous frameworks focus on sizing the numerical architectural hyper-parameters while neglect searching the PE connectivities and compiler mappings. To tackle this challenge, we propose Neural Accelerator Architecture Search (NAAS) which holistically searches the neural network architecture, accelerator architecture, and compiler mapping in one optimization loop. NAAS composes highly matched architectures together with efficient mapping. As a data-driven approach, NAAS rivals the human design Eyeriss by 4.4x EDP reduction with 2.7% accuracy improvement on ImageNet under the same computation resource, and offers 1.4x to 3.5x EDP reduction than only sizing the architectural hyper-parameters.