# HAO: Hardware-aware neural Architecture Optimization for Efficient Inference

arXiv: https://arxiv.org/abs/2104.12766 · HTML: https://mlsanity.com/paper/2104.12766/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 55 references that are in the map.
- First posted on arXiv 26 Apr 2021.

## Builds on (most cited first, 30 of 37)

- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016)
- [ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design](https://mlsanity.com/paper/1807.11164/) (2018)
- [SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size](https://mlsanity.com/paper/1602.07360/) (2016)
- [MnasNet: Platform-Aware Neural Architecture Search for Mobile](https://mlsanity.com/paper/1807.11626/) (2018)
- [Regularized Evolution for Image Classifier Architecture Search](https://mlsanity.com/paper/1802.01548/) (2018)
- [Efficient Neural Architecture Search via Parameter Sharing](https://mlsanity.com/paper/1802.03268/) (2018)
- [ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware](https://mlsanity.com/paper/1812.00332/) (2018)
- [Progressive Neural Architecture Search](https://mlsanity.com/paper/1712.00559/) (2017)
- [FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search](https://mlsanity.com/paper/1812.03443/) (2018)
- [HAQ: Hardware-Aware Automated Quantization with Mixed Precision](https://mlsanity.com/paper/1811.08886/) (2018)
- [HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precision](https://mlsanity.com/paper/1905.03696/) (2019)
- [LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks](https://mlsanity.com/paper/1807.10029/) (2018)
- [ZeroQ: A Novel Zero Shot Quantization Framework](https://mlsanity.com/paper/2001.00281/) (2020)
- [HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networks](https://mlsanity.com/paper/1911.03852/) (2019)
- [Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights](https://mlsanity.com/paper/1702.03044/) (2017)
- [Mixed Precision Quantization of ConvNets via Differentiable Neural Architecture Search](https://mlsanity.com/paper/1812.00090/) (2018)
- [Neural Predictor for Neural Architecture Search](https://mlsanity.com/paper/1912.00848/) (2019)
- [Adaptive Quantization for Deep Neural Network](https://mlsanity.com/paper/1712.01048/) (2017)
- [HAWQV3: Dyadic Neural Network Quantization](https://mlsanity.com/paper/2011.10680/) (2020)
- [Hardware/Software Co-Exploration of Neural Architectures](https://mlsanity.com/paper/1907.04650/) (2019)
- [FPGA/DNN Co-Design: An Efficient Design Methodology for IoT Intelligence on the Edge](https://mlsanity.com/paper/1904.04421/) (2019)
- [Accuracy vs. Efficiency: Achieving Both through FPGA-Implementation Aware Neural Architecture Search](https://mlsanity.com/paper/1901.11211/) (2019)
- [EDD: Efficient Differentiable DNN Architecture and Implementation Co-search for Embedded AI Solutions](https://mlsanity.com/paper/2005.02563/) (2020)
- [Best of Both Worlds: AutoML Codesign of a CNN and its Hardware Accelerator](https://mlsanity.com/paper/2002.05022/) (2020)

## Built on by (most cited first, 16 of 16)

- [Applications and Techniques for Fast Machine Learning in Science](https://mlsanity.com/paper/2110.13041/) (2021)
- [Vision Transformers on the Edge: A Comprehensive Survey of Model Compression and Acceleration Strategies](https://mlsanity.com/paper/2503.02891/) (2025)
- [AnalogNAS: A Neural Network Design Framework for Accurate Inference with Analog In-Memory Computing](https://mlsanity.com/paper/2305.10459/) (2023)
- [Analysis of Quantization on MLP-based Vision Models](https://mlsanity.com/paper/2209.06383/) (2022)
- [M4BRAM: Mixed-Precision Matrix-Matrix Multiplication in FPGA Block RAMs](https://mlsanity.com/paper/2311.02758/) (2023)
- [Achieving Peak Performance for Large Language Models: A Systematic Review](https://mlsanity.com/paper/2409.04833/) (2024)
- [MetaML-Pro: Cross-Stage Design Flow Automation for Efficient Deep Learning Acceleration](https://mlsanity.com/paper/2502.05850/) (2025)
- [MetaML: Automating Customizable Cross-Stage Design-Flow for Deep Learning Acceleration](https://mlsanity.com/paper/2306.08746/) (2023)
- [DeepBurning-MixQ: An Open Source Mixed-Precision Neural Network Accelerator Design Framework for FPGAs](https://mlsanity.com/paper/2308.11334/) (2023)
- [How to Reach Real-Time AI on Consumer Devices? Solutions for Programmable and Custom Architectures](https://mlsanity.com/paper/2106.15021/) (2021)
- [Algorithm and Hardware Co-design for Reconfigurable CNN Accelerator](https://mlsanity.com/paper/2111.12787/) (2021)
- [DyBit: Dynamic Bit-Precision Numbers for Efficient Quantized Neural Network Inference](https://mlsanity.com/paper/2302.12510/) (2023)
- [End-to-end codesign of Hessian-aware quantized neural networks for FPGAs and ASICs](https://mlsanity.com/paper/2304.06745/) (2023)
- [End-to-End Neural Network Compression via fracℓ1ℓ2 Regularized Latency Surrogates](https://mlsanity.com/paper/2306.05785/) (2023)
- [Continuous-Flow Data-Rate-Aware CNN Inference on FPGA](https://mlsanity.com/paper/2601.19940/) (2026)
- [UnrealNAS: Can We Search Neural Architectures with Unreal Data?](https://mlsanity.com/paper/2205.02162/) (2022)

## Abstract (arXiv)

Automatic algorithm-hardware co-design for DNN has shown great success in improving the performance of DNNs on FPGAs. However, this process remains challenging due to the intractable search space of neural network architectures and hardware accelerator implementation. Differing from existing hardware-aware neural architecture search (NAS) algorithms that rely solely on the expensive learning-based approaches, our work incorporates integer programming into the search algorithm to prune the design space. Given a set of hardware resource constraints, our integer programming formulation directly outputs the optimal accelerator configuration for mapping a DNN subgraph that minimizes latency. We use an accuracy predictor for different DNN subgraphs with different quantization schemes and generate accuracy-latency pareto frontiers. With low computational cost, our algorithm can generate quantized networks that achieve state-of-the-art accuracy and hardware performance on Xilinx Zynq (ZU3EG) FPGA for image classification on ImageNet dataset. The solution searched by our algorithm achieves 72.5% top-1 accuracy on ImageNet at framerate 50, which is 60% faster than MnasNet and 135% faster than FBNet with comparable accuracy.