# Simultaneous Approximation of the Score Function and Its Derivatives by Deep Neural Networks

arXiv: https://arxiv.org/abs/2512.23643 · HTML: https://mlsanity.com/paper/2512.23643/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 49 references that are in the map.
- First posted on arXiv 29 Dec 2025.

## Builds on (most cited first, 30 of 33)

- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Video Diffusion Models](https://mlsanity.com/paper/2204.03458/) (2022)
- [DiffWave: A Versatile Diffusion Model for Audio Synthesis](https://mlsanity.com/paper/2009.09761/) (2020)
- [Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions](https://mlsanity.com/paper/2209.11215/) (2022)
- [The Intrinsic Dimension of Images and Its Impact on Learning](https://mlsanity.com/paper/2104.08894/) (2021)
- [Convergence of denoising diffusion models under the manifold hypothesis](https://mlsanity.com/paper/2208.05314/) (2022)
- [Nearly d-Linear Convergence Bounds for Diffusion Models via Stochastic Localization](https://mlsanity.com/paper/2308.03686/) (2023)
- [Hidden Physics Models: Machine Learning of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1708.00588/) (2017)
- [Score Approximation, Estimation and Distribution Recovery of Diffusion Models on Low-Dimensional Data](https://mlsanity.com/paper/2302.07194/) (2023)
- [Diffusion Models are Minimax Optimal Distribution Estimators](https://mlsanity.com/paper/2303.01861/) (2023)
- [Sobolev Training for Neural Networks](https://mlsanity.com/paper/1706.04859/) (2017)
- [Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating PDEs](https://mlsanity.com/paper/2006.16144/) (2020)
- [On the approximation of functions by tanh neural networks](https://mlsanity.com/paper/2104.08938/) (2021)
- [Accelerating Convergence of Score-Based Diffusion Models, Provably](https://mlsanity.com/paper/2403.03852/) (2024)
- [Deep ReLU network approximation of functions on a manifold](https://mlsanity.com/paper/1908.00695/) (2019)
- [A Sharp Convergence Theory for The Probability Flow ODEs of Diffusion Models](https://mlsanity.com/paper/2408.02320/) (2024)
- [Convergence of Diffusion Models Under the Manifold Hypothesis in High-Dimensions](https://mlsanity.com/paper/2409.18804/) (2024)
- [Verifying the Union of Manifolds Hypothesis for Image Data](https://mlsanity.com/paper/2207.02862/) (2022)
- [Adapting to Unknown Low-Dimensional Structures in Score-Based Diffusion Models](https://mlsanity.com/paper/2405.14861/) (2024)
- [Non-asymptotic bounds for sampling algorithms without log-concavity](https://mlsanity.com/paper/1808.07105/) (2018)
- [Simultaneous approximation of a smooth function and its derivatives by deep neural networks with piecewise-polynomial activations](https://mlsanity.com/paper/2206.09527/) (2022)
- [Statistical guarantees for generative models without domination](https://mlsanity.com/paper/2010.09237/) (2020)
- [Generalization error bound for denoising score matching under relaxed manifold assumption](https://mlsanity.com/paper/2502.13662/) (2025)
- [Faster Diffusion Models via Higher-Order Approximation](https://mlsanity.com/paper/2506.24042/) (2025)
- [Approximations with deep neural networks in Sobolev time-space](https://mlsanity.com/paper/2101.06115/) (2020)
- [Differentiable Neural Networks with RePU Activation: with Applications to Score Estimation and Isotonic Regression](https://mlsanity.com/paper/2305.00608/) (2023)
- [Fast Convergence for High-Order ODE Solvers in Diffusion Probabilistic Models](https://mlsanity.com/paper/2506.13061/) (2025)

## Built on by (most cited first, 3 of 3)

- [Inference-Time Alignment for Diffusion Models via Variationally Stable Doob's Matching](https://mlsanity.com/paper/2601.06514/) (2026)
- [A Quantitative Approximation Framework for Flow Distillation in Diffusion Models](https://mlsanity.com/paper/2606.03820/) (2026)
- [Multi-Source Transfer Learning of Sparse Single-Index Models](https://mlsanity.com/paper/2606.29658/) (2026)

## Abstract (arXiv)

We present a theory for simultaneous approximation of the score function and its derivatives, enabling the handling of data distributions with low-dimensional structure and unbounded support. Our approximation error bounds match those in the literature while relying on assumptions that relax the usual bounded support requirement. Crucially, our bounds are free from the curse of dimensionality. Moreover, we establish approximation guarantees for derivatives of any prescribed order, extending beyond the commonly considered first-order setting.