# Discretization-Aware Architecture Search

arXiv: https://arxiv.org/abs/2007.03154 · HTML: https://mlsanity.com/paper/2007.03154/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Jul 2020.

## Built on by (most cited first, 14 of 14)

- [DiNTS: Differentiable Neural Network Topology Search for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.15954/) (2021)
- [HyperSegNAS: Bridging One-Shot Neural Architecture Search with 3D Medical Image Segmentation using HyperNet](https://mlsanity.com/paper/2112.10652/) (2021)
- [Auto-NBA: Efficient and Effective Search Over the Joint Space of Networks, Bitwidths, and Accelerators](https://mlsanity.com/paper/2106.06575/) (2021)
- [Improving Differentiable Architecture Search via Self-Distillation](https://mlsanity.com/paper/2302.05629/) (2023)
- [QuantNAS for super resolution: searching for efficient quantization-friendly architectures against quantization noise](https://mlsanity.com/paper/2208.14839/) (2022)
- [Differentiable Convolution Search for Point Cloud Processing](https://mlsanity.com/paper/2108.12856/) (2021)
- [TPAD: Identifying Effective Trajectory Predictions Under the Guidance of Trajectory Anomaly Detection Model](https://mlsanity.com/paper/2201.02941/) (2022)
- [DARTSRepair: Core-failure-set Guided DARTS for Network Robustness to Common Corruptions](https://mlsanity.com/paper/2209.10381/) (2022)
- [MS-DARTS: Mean-Shift Based Differentiable Architecture Search](https://mlsanity.com/paper/2108.09996/) (2021)
- [Exploring Complicated Search Spaces with Interleaving-Free Sampling](https://mlsanity.com/paper/2112.02488/) (2021)
- [The devil is in discretization discrepancy. Robustifying Differentiable NAS with Single-Stage Searching Protocol](https://mlsanity.com/paper/2405.16610/) (2024)
- [S2HPruner: Soft-to-Hard Distillation Bridges the Discretization Gap in Pruning](https://mlsanity.com/paper/2410.07046/) (2024)
- [FX-DARTS: Designing Topology-unconstrained Architectures with Differentiable Architecture Search and Entropy-based Super-network Shrinking](https://mlsanity.com/paper/2504.20079/) (2025)
- [DNAD: Differentiable Neural Architecture Distillation](https://mlsanity.com/paper/2504.20080/) (2025)

## Abstract (arXiv)

The search cost of neural architecture search (NAS) has been largely reduced by weight-sharing methods. These methods optimize a super-network with all possible edges and operations, and determine the optimal sub-network by discretization, textiti.e., pruning off weak candidates. The discretization process, performed on either operations or edges, incurs significant inaccuracy and thus the quality of the final architecture is not guaranteed. This paper presents discretization-aware architecture search (DAtextsuperscript2S), with the core idea being adding a loss term to push the super-network towards the configuration of desired topology, so that the accuracy loss brought by discretization is largely alleviated. Experiments on standard image classification benchmarks demonstrate the superiority of our approach, in particular, under imbalanced target network configurations that were not studied before.