# Differentiable Architecture Search with Ensemble Gumbel-Softmax

arXiv: https://arxiv.org/abs/1905.01786 · HTML: https://mlsanity.com/paper/1905.01786/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 May 2019.

## Built on by (most cited first, 6 of 6)

- [Spatial Attention-based Implicit Neural Representation for Arbitrary Reduction of MRI Slice Spacing](https://mlsanity.com/paper/2205.11346/) (2022)
- [Analytical Uncertainty-Based Loss Weighting in Multi-Task Learning](https://mlsanity.com/paper/2408.07985/) (2024)
- [Learning a Consensus Sub-Network with Polarization Regularization and One Pass Training](https://mlsanity.com/paper/2302.10798/) (2023)
- [Mutually-aware Sub-Graphs Differentiable Architecture Search](https://mlsanity.com/paper/2107.04324/) (2021)
- [Straight Through Gumbel Softmax Estimator based Bimodal Neural Architecture Search for Audio-Visual Deepfake Detection](https://mlsanity.com/paper/2406.13384/) (2024)
- [GroupMask: Layer-Adaptive Group-wise Sparsity for Semi-Structured LLM Pruning](https://mlsanity.com/paper/2609.33977/) (2026)

## Abstract (arXiv)

For network architecture search (NAS), it is crucial but challenging to simultaneously guarantee both effectiveness and efficiency. Towards achieving this goal, we develop a differentiable NAS solution, where the search space includes arbitrary feed-forward network consisting of the predefined number of connections. Benefiting from a proposed ensemble Gumbel-Softmax estimator, our method optimizes both the architecture of a deep network and its parameters in the same round of backward propagation, yielding an end-to-end mechanism of searching network architectures. Extensive experiments on a variety of popular datasets strongly evidence that our method is capable of discovering high-performance architectures, while guaranteeing the requisite efficiency during searching.