# Hardware/Software Co-Exploration of Neural Architectures

arXiv: https://arxiv.org/abs/1907.04650 · HTML: https://mlsanity.com/paper/1907.04650/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jul 2019.

## Built on by (most cited first, 30 of 50)

- [A Comprehensive Survey on Hardware-Aware Neural Architecture Search](https://mlsanity.com/paper/2101.09336/) (2021)
- [Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications](https://mlsanity.com/paper/2204.11786/) (2022)
- [Machine Learning for Microcontroller-Class Hardware: A Review](https://mlsanity.com/paper/2205.14550/) (2022)
- [Green Edge AI: A Contemporary Survey](https://mlsanity.com/paper/2312.00333/) (2023)
- [NAAS: Neural Accelerator Architecture Search](https://mlsanity.com/paper/2105.13258/) (2021)
- [Enabling On-Device Self-Supervised Contrastive Learning With Selective Data Contrast](https://mlsanity.com/paper/2106.03796/) (2021)
- [The Larger The Fairer? Small Neural Networks Can Achieve Fairness for Edge Devices](https://mlsanity.com/paper/2202.11317/) (2022)
- [MPCViT: Searching for Accurate and Efficient MPC-Friendly Vision Transformer with Heterogeneous Attention](https://mlsanity.com/paper/2211.13955/) (2022)
- [A Full-Stack Search Technique for Domain Optimized Deep Learning Accelerators](https://mlsanity.com/paper/2105.12842/) (2021)
- [Uncertainty Modeling of Emerging Device-based Computing-in-Memory Neural Accelerators with Application to Neural Architecture Search](https://mlsanity.com/paper/2107.06871/) (2021)
- [HAO: Hardware-aware neural Architecture Optimization for Efficient Inference](https://mlsanity.com/paper/2104.12766/) (2021)
- [G-CoS: GNN-Accelerator Co-Search Towards Both Better Accuracy and Efficiency](https://mlsanity.com/paper/2109.08983/) (2021)
- [Computation-efficient Deep Learning for Computer Vision: A Survey](https://mlsanity.com/paper/2308.13998/) (2023)
- [Automated Deep Learning: Neural Architecture Search Is Not the End](https://mlsanity.com/paper/2112.09245/) (2021)
- [Rethinking Co-design of Neural Architectures and Hardware Accelerators](https://mlsanity.com/paper/2102.08619/) (2021)
- [Auto-NBA: Efficient and Effective Search Over the Joint Space of Networks, Bitwidths, and Accelerators](https://mlsanity.com/paper/2106.06575/) (2021)
- [SECDA: Efficient Hardware/Software Co-Design of FPGA-based DNN Accelerators for Edge Inference](https://mlsanity.com/paper/2110.00478/) (2021)
- [Exploration of Quantum Neural Architecture by Mixing Quantum Neuron Designs](https://mlsanity.com/paper/2109.03806/) (2021)
- [On the Viability of using LLMs for SW/HW Co-Design: An Example in Designing CiM DNN Accelerators](https://mlsanity.com/paper/2306.06923/) (2023)
- [On-Device Unsupervised Image Segmentation](https://mlsanity.com/paper/2303.12753/) (2023)
- [Dancing along Battery: Enabling Transformer with Run-time Reconfigurability on Mobile Devices](https://mlsanity.com/paper/2102.06336/) (2021)
- [RADARS: Memory Efficient Reinforcement Learning Aided Differentiable Neural Architecture Search](https://mlsanity.com/paper/2109.05691/) (2021)
- [CATransformers: Carbon Aware Transformers Through Joint Model-Hardware Optimization](https://mlsanity.com/paper/2505.01386/) (2025)
- [CODEBench: A Neural Architecture and Hardware Accelerator Co-Design Framework](https://mlsanity.com/paper/2212.03965/) (2022)
- [MATCH: Model-Aware TVM-based Compilation for Heterogeneous Edge Devices](https://mlsanity.com/paper/2410.08855/) (2024)
- [MetaML-Pro: Cross-Stage Design Flow Automation for Efficient Deep Learning Acceleration](https://mlsanity.com/paper/2502.05850/) (2025)
- [RBFleX-NAS: Training-Free Neural Architecture Search Using Radial Basis Function Kernel and Hyperparameter Detection](https://mlsanity.com/paper/2503.22733/) (2025)
- [A3C-S: Automated Agent Accelerator Co-Search towards Efficient Deep Reinforcement Learning](https://mlsanity.com/paper/2106.06577/) (2021)
- [RT-DNAS: Real-time Constrained Differentiable Neural Architecture Search for 3D Cardiac Cine MRI Segmentation](https://mlsanity.com/paper/2206.04682/) (2022)
- [MetaML: Automating Customizable Cross-Stage Design-Flow for Deep Learning Acceleration](https://mlsanity.com/paper/2306.08746/) (2023)

## Abstract (arXiv)

We propose a novel hardware and software co-exploration framework for efficient neural architecture search (NAS). Different from existing hardware-aware NAS which assumes a fixed hardware design and explores the neural architecture search space only, our framework simultaneously explores both the architecture search space and the hardware design space to identify the best neural architecture and hardware pairs that maximize both test accuracy and hardware efficiency. Such a practice greatly opens up the design freedom and pushes forward the Pareto frontier between hardware efficiency and test accuracy for better design tradeoffs. The framework iteratively performs a two-level (fast and slow) exploration. Without lengthy training, the fast exploration can effectively fine-tune hyperparameters and prune inferior architectures in terms of hardware specifications, which significantly accelerates the NAS process. Then, the slow exploration trains candidates on a validation set and updates a controller using the reinforcement learning to maximize the expected accuracy together with the hardware efficiency. Experiments on ImageNet show that our co-exploration NAS can find the neural architectures and associated hardware design with the same accuracy, 35.24% higher throughput, 54.05% higher energy efficiency and 136x reduced search time, compared with the state-of-the-art hardware-aware NAS.