Optimal Tradeoffs Between Network Size and Parameter Magnitude in Neural Approximation and Minimax Regression

The statistical accuracy of neural networks depends on both their approximation power and the complexity of the class fitted from data. While increasing network size is a natural way to improve approximation, parameter magnitude provides another resource whose role must be quantified in both respects. We establish a sharp width--magnitude tradeoff at fixed depth using one elementary bounded 1-Lipschitz Dyadic--Triangular Activation. For the unit β-H"older ball on [0,1]d with 0<β≤1, the optimal Lp approximation error for 0<p<∞ is of order [N2log(eNT)]-β/d when the network width satisfies N≥2d+3 and the parameter magnitudes are bounded by T≥1. Matching lower bounds hold for every fixed globally H"older activation; its H"older exponent affects the constants but not the rate. Under bounded design densities and independent centered sub-Gaussian noise, approximate least squares over the full clipped class at depth 23 attains the classical H"older minimax risk O(M^-frac2β2β+d) without logarithmic loss whenever N2log(eNT)asymp M^fracd2β+d, where M is the sample size. This yields a continuum of statistically optimal choices, ranging from unit parameter radius to fixed network size. At fixed size, four hidden layers with at most 8d+7 nonzero parameters give a near-optimal radius, while six layers with at most 8d+27 attain the optimal order log T=O(η-d/β) at approximation error η. The same decoding method also yields fixed-size Transformer approximation.