CAGE-NAS: Certified Functional Descent for Efficient Model Growth
Not on the map. Why?
Abstract (arXiv)
The progressive growth of neural networks requires deciding when the current representation remains sufficient for optimization and when it should be expanded. CAGE-NAS formulates this decision in function space through an admissibility criterion on approximations of the functional gradient. As long as a representation enables a certified Functional Gradient Descent step, the architecture remains fixed; when the criterion fails, a function-preserving expansion is applied and the resulting representation is evaluated again. As the main instance, we study the family induced by the tangent space, using a regularized projection of the functional gradient. In a controlled setting with exact certification, CAGE-NAS produces architectures positioned above the 99.8th performance percentile by held-out RMSE among all admissible alternatives within the same parameter budget, without enumerating them during the growth trajectory.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 8 of its 11 references that are in the map. First posted on arXiv 1 Oct 2026.
Builds on
8 of 11 references.
References · 8
- 2021Explaining Neural Scaling Lawscited by 353
- 2022GradMax: Growing Neural Networks using Gradient Informationcited by 64
- 2021Firefly Neural Architecture Descent: a General Approach for Growing Neural Networkscited by 48
- 2022When, where, and how to add new neurons to ANNscited by 18
- 2024Growing Tiny Networks: Spotting Expressivity Bottlenecks and Fixing Them Optimallycited by 9
- 2024When To Grow? A Fitting Risk-Aware Policy for Layer Growing in Deep Neural Networkscited by 3
- 2025Growth strategies for arbitrary DAG neural architecturescited by 2
- 2026Functional Gradient Descent with Adaptive Representationscited by 1
Built on by
Not cited yet.