# Growth strategies for arbitrary DAG neural architectures

arXiv: https://arxiv.org/abs/2501.12690 · HTML: https://mlsanity.com/paper/2501.12690/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 8 references that are in the map.
- First posted on arXiv 22 Jan 2025.

## Builds on (most cited first, 5 of 5)

- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [GradMax: Growing Neural Networks using Gradient Information](https://mlsanity.com/paper/2201.05125/) (2022)
- [Firefly Neural Architecture Descent: a General Approach for Growing Neural Networks](https://mlsanity.com/paper/2102.08574/) (2021)
- [When, where, and how to add new neurons to ANNs](https://mlsanity.com/paper/2202.08539/) (2022)
- [Growing Tiny Networks: Spotting Expressivity Bottlenecks and Fixing Them Optimally](https://mlsanity.com/paper/2405.19816/) (2024)

## Built on by (most cited first, 2 of 2)

- [Neuron-Level Architecture Growth: A Controlled Evaluation for EEG Time-Series Decoding](https://mlsanity.com/paper/2609.33880/) (2026)
- [CAGE-NAS: Certified Functional Descent for Efficient Model Growth](https://mlsanity.com/paper/2610.01173/) (2026)

## Abstract (arXiv)

Deep learning has shown impressive results obtained at the cost of training huge neural networks. However, the larger the architecture, the higher the computational, financial, and environmental costs during training and inference. We aim at reducing both training and inference durations. We focus on Neural Architecture Growth, which can increase the size of a small model when needed, directly during training using information from the backpropagation. We expand existing work and freely grow neural networks in the form of any Directed Acyclic Graph by reducing expressivity bottlenecks in the architecture. We explore strategies to reduce excessive computations and steer network growth toward more parameter-efficient architectures.