# Representation Costs in Data Science: Foundations and the Quasi-Banach Spaces of Deep Neural Networks

arXiv: https://arxiv.org/abs/2606.14954 · HTML: https://mlsanity.com/paper/2606.14954/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 163 references that are in the map.
- First posted on arXiv 12 Jun 2026.

## Builds on (most cited first, 5 of 5)

- [Lower bounds over Boolean inputs for deep neural networks with ReLU gates](https://mlsanity.com/paper/1711.03073/) (2017)
- [Neural reproducing kernel Banach spaces and representer theorems for deep networks](https://mlsanity.com/paper/2403.08750/) (2024)
- [Deep Networks are Reproducing Kernel Chains](https://mlsanity.com/paper/2501.03697/) (2025)
- [On the inductive bias of infinite-depth ResNets and the bottleneck rank](https://mlsanity.com/paper/2501.19149/) (2025)
- [Learning Sparse Compositional Functions with Norm-Constrained Neural Networks](https://mlsanity.com/paper/2605.25608/) (2026)

## Built on by (most cited first, 2 of 2)

- [Deep Neural Variation Spaces: A Unifying Perspective on Depth and Complexity](https://mlsanity.com/paper/2607.05546/) (2026)
- [Variation Brownian Kernel Ladders](https://mlsanity.com/paper/2608.13882/) (2026)

## Abstract (arXiv)

We develop a general framework for analyzing representation costs induced by parameter-space regularizers in data-fitting methods. For an arbitrary parametric method, we define its representation cost and native function space, prove existence, and identify conditions under which parameter-space and function-space problems have equal infimal values and minimizers transfer between them. This framework yields representer theorems and recovers classical formulations---including kernel methods and RKHSs, wavelets and Besov spaces, and shallow neural networks and variation spaces---as special cases. Our main new results concern depth-L feedforward ReLU networks with weight-decay regularization. For these networks, we prove that the representation cost is a power of a quasi-seminorm and that, under suitable hypotheses, the native space is a quasi-Banach space with nonconvex unit ball when L > 2. These results identify a novel depth-dependent quasi-Banach geometry induced by weight decay.