# Sharp Approximation Rates for Neural Networks with Affine Latent Parameterizations

arXiv: https://arxiv.org/abs/2608.31157 · HTML: https://mlsanity.com/paper/2608.31157/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 98 references that are in the map.
- First posted on arXiv 31 Aug 2026.

## Builds on (most cited first, 8 of 8)

- [On the Activation Function Dependence of the Spectral Bias of Neural Networks](https://mlsanity.com/paper/2208.04924/) (2022)
- [Deep Neural Network Approximation of Invariant Functions through Dynamical Systems](https://mlsanity.com/paper/2208.08707/) (2022)
- [On the Universal Approximation Property of Deep Fully Convolutional Neural Networks](https://mlsanity.com/paper/2211.14047/) (2022)
- [Multigrade Neural Network Approximation](https://mlsanity.com/paper/2601.16884/) (2026)
- [Deep learning and the rate of approximation by flows](https://mlsanity.com/paper/2603.15363/) (2026)
- [Layer-wise Geometric Approximation Rates for Deep Networks](https://mlsanity.com/paper/2604.20219/) (2026)
- [Kilobyte Models: Neural Networks as a Seed and a Quantized Latent](https://mlsanity.com/paper/2608.00860/) (2026)
- [Do Neural Networks Really Beat the Curse of Dimensionality? A Bit-Complexity View](https://mlsanity.com/paper/2608.01357/) (2026)

## Abstract (arXiv)

Many parameter-efficient methods generate the parameters of a large neural network from a low-dimensional latent representation. Given an architecture Phi with P_Phi parameter slots, we write boldsymbolθf=G(boldsymbolxif), where GcolonℝM→ℝP_Phi is a parameter generator and boldsymbolxif∈ℝM is a latent representation of the target function f. The architecture Phi and the generator G are shared across the entire target class, while each target f is represented by its own latent vector boldsymbolxif, with Phi_G(boldsymbolxif) approximating f. This framework encompasses hypernetworks, low-dimensional parameterizations, parameter-efficient adaptation, and model compression. Understanding the tradeoff between the latent dimension M and the network budget P is therefore fundamental to characterizing the expressive efficiency of these methods. We study this tradeoff for affine generators and fully connected ReLU architectures. More precisely, optimizing jointly over architectures Phi satisfying P_Phi≤ P and affine generators G:ℝM→ ℝP_Phi, we prove that the optimal worst-case uniform approximation error over the unit ball of α-H"older functions on [0,1]d, where 0<α≤1, has the sharp order  bigl(PminM,Pbigr)-α/d.  In particular, our result shows that even a fixed-dimensional latent space suffices to achieve vanishing approximation error as the network budget increases.