What Does Model Growth Really Add? Functional Capacity Beyond Parameter Count

Model growth adds parameters to a trained checkpoint and continues training, aiming to increase capacity while reusing the existing model instead of training from scratch. But does growing a trained model actually create new functional capacity that survives subsequent training? Parameter count alone cannot answer this question, and ``capacity'' is itself ambiguous; it can refer to parameter count, effective dimensionality, or the ability to represent new functions. We measure it directly, as the number of new, functionally independent directions a growth step adds, formalized as the rank increment of the model's functional Jacobian and estimated matrix-free at scale. For zero-gated function-preserving growth, this increment is exactly K under a transversality condition, i.e. operatornamerank(Jgrown)=operatornamerank(Jold)+K. Continually training a Transformer grown from 253M to 857M parameters across three domains, we find these added directions persist, with all K=36 remaining independent after continual learning, even when the model catastrophically forgets (perplexity 28→481) or we deliberately destroy the old function (25.9→127). Geometric capacity and retained function are therefore decoupled. The functional capacity introduced by growth persists, while the previously learned function is not retained. A second, geometrically distinct mechanism, Net2Net, reaches the same K-dimensional capacity from directions that are dormant at initialization (0 → K) and then retains it, so the decoupling is not specific to the zero-gate parameterization. The implication is precise. Catastrophic forgetting need not reflect a loss of functional capacity, and preserving dimensionality alone is insufficient to preserve the learned function.