# Learning Theory of Distribution Regression with Neural Networks

arXiv: https://arxiv.org/abs/2307.03487 · HTML: https://mlsanity.com/paper/2307.03487/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 45 references that are in the map.
- First posted on arXiv 7 Jul 2023.

## Builds on (most cited first, 15 of 15)

- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [Error bounds for approximations with deep ReLU networks](https://mlsanity.com/paper/1610.01145/) (2016)
- [Nonparametric regression using deep neural networks with ReLU activation function](https://mlsanity.com/paper/1708.06633/) (2017)
- [Universality of Deep Convolutional Neural Networks](https://mlsanity.com/paper/1805.10769/) (2018)
- [Deep Network Approximation for Smooth Functions](https://mlsanity.com/paper/2001.03040/) (2020)
- [Optimal Approximation with Sparsely Connected Deep Neural Networks](https://mlsanity.com/paper/1705.01714/) (2017)
- [Learning Theory for Distribution Regression](https://mlsanity.com/paper/1411.2066/) (2014)
- [Nonlinear Approximation and (Deep) ReLU Networks](https://mlsanity.com/paper/1905.02199/) (2019)
- [Theory of Deep Convolutional Neural Networks III: Approximating Radial Functions](https://mlsanity.com/paper/2107.00896/) (2021)
- [Distribution-Free Distribution Regression](https://mlsanity.com/paper/1302.0082/) (2013)
- [A Functional Perspective on Learning Symmetric Functions with Neural Networks](https://mlsanity.com/paper/2008.06952/) (2020)
- [Stochastic Deep Networks](https://mlsanity.com/paper/1811.07429/) (2018)
- [Deep Neural Networks for Rotation-Invariance Approximation and Learning](https://mlsanity.com/paper/1904.01814/) (2019)
- [Robust Kernel-based Distribution Regression](https://mlsanity.com/paper/2104.10637/) (2021)
- [Stochastic Gradient Descent Meets Distribution Regression](https://mlsanity.com/paper/2010.12842/) (2020)

## Built on by (most cited first, 3 of 3)

- [Beyond Universal Approximation Theorems: Algorithmic Uniform Approximation by Neural Networks Trained with Noisy Data](https://mlsanity.com/paper/2509.00924/) (2025)
- [Learning Theory of Transformers: Local-to-Global Approximation via Softmax Partition of Unity](https://mlsanity.com/paper/2605.08811/) (2026)
- [Generalization Analysis of Transformers in Distribution Regression](https://mlsanity.com/paper/2606.29256/) (2026)

## Abstract (arXiv)

In this paper, we aim at establishing an approximation theory and a learning theory of distribution regression via a fully connected neural network (FNN). In contrast to the classical regression methods, the input variables of distribution regression are probability measures. Then we often need to perform a second-stage sampling process to approximate the actual information of the distribution. On the other hand, the classical neural network structure requires the input variable to be a vector. When the input samples are probability distributions, the traditional deep neural network method cannot be directly used and the difficulty arises for distribution regression. A well-defined neural network structure for distribution inputs is intensively desirable. There is no mathematical model and theoretical analysis on neural network realization of distribution regression. To overcome technical difficulties and address this issue, we establish a novel fully connected neural network framework to realize an approximation theory of functionals defined on the space of Borel probability measures. Furthermore, based on the established functional approximation results, in the hypothesis space induced by the novel FNN structure with distribution inputs, almost optimal learning rates for the proposed distribution regression model up to logarithmic terms are derived via a novel two-stage error decomposition technique.