mlsanityData to 14 Oct 2026

AutoGrow: Automatic Layer Growing in Deep Convolutional Networks

arXiv 1906.02909 ↗First posted 7 Jun 2019cs.LG cs.CV cs.NEcited by 26

Not on the map. Why?

Abstract (arXiv)

Depth is a key component of Deep Neural Networks (DNNs), however, designing depth is heuristic and requires many human efforts. We propose AutoGrow to automate depth discovery in DNNs: starting from a shallow seed architecture, AutoGrow grows new layers if the growth improves the accuracy; otherwise, stops growing and thus discovers the depth. We propose robust growing and stopping policies to generalize to different network architectures and datasets. Our experiments show that by applying the same policy to different network architectures, AutoGrow can always discover near-optimal depth on various datasets of MNIST, FashionMNIST, SVHN, CIFAR10, CIFAR100 and ImageNet. For example, in terms of accuracy-computation trade-off, AutoGrow discovers a better depth combination in ResNets than human experts. Our AutoGrow is efficient. It discovers depth within similar time of training a single DNN. Our code is available at https://github.com/wenwei202/autogrow.

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). First posted on arXiv 7 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 7; 2022: 4; 2023: 5; 2024: 6; 2025: 3; 2026: 1.

26 papers.

Sort:

By area:

  1. 2022GradMax: Growing Neural Networks using Gradient Informationcited by 66
  2. 2021Firefly Neural Architecture Descent: a General Approach for Growing Neural Networkscited by 49
  3. 2022Automated Progressive Learning for Efficient Training of Vision Transformerscited by 37
  4. 2023Masked Structural Growth for 2x Faster Language Model Pre-trainingcited by 33
  5. 2021On Anytime Learning at Macroscalecited by 20
  6. 2022When, where, and how to add new neurons to ANNscited by 18
  7. 2021GROWN: GRow Only When Necessary for Continual Learningcited by 11
  8. 2022DyRep: Bootstrapping Training with Dynamic Re-parameterizationcited by 11
  9. 2023Accelerated Training via Incrementally Growing Neural Networks using Variance Transfer and Learning Rate Adaptationcited by 8
  10. 2023MixtureGrowth: Growing Neural Networks by Recombining Learned Parameterscited by 7
  11. 2024Efficient Training of Large Vision Models via Advanced Automated Progressive Learningcited by 5
  12. 2025LESA: Learnable LLM Layer Scaling-Upcited by 5
14 more
  1. 2021Transformed CNNs: recasting pre-trained convolutional layers with self-attentioncited by 4
  2. 2024When To Grow? A Fitting Risk-Aware Policy for Layer Growing in Deep Neural Networkscited by 3
  3. 2023Shrink-Perturb Improves Architecture Mixing during Population Based Training for Neural Architecture Searchcited by 2
  4. 2024A General and Efficient Training for Transformer via Token Expansioncited by 2
  5. 2025Topological derivative approach for deep neural network architecture adaptationcited by 2
  6. 2021Compact CNN Structure Learning by Knowledge Distillationcited by 1
  7. 2023SensLI: Sensitivity-Based Layer Insertion for Neural Networkscited by 1
  8. 2024Overcoming Growth-Induced Forgetting in Task-Agnostic Continual Learningcited by 1
  9. 2025Growing with Experience: Growing Neural Networks in Deep Reinforcement Learningcited by 1
  10. 2021Differentiable Network Adaption with Elastic Search Spacecited by 0
  11. 2021Compact Multi-level Sparse Neural Networks with Input Independent Dynamic Reroutingcited by 0
  12. 2024A Multi-Level Framework for Accelerating Training Transformer Modelscited by 0
  13. 2024Growing Efficient Accurate and Robust Neural Networks on the Edgecited by 0
  14. 2026Quantifying Depth Sufficiency in Residual Neural Networks: A First-Order Criterioncited by 0