Data to 7 Oct 2026

High-dimensional Asymptotics of Denoising Autoencoders

arXiv 2305.11041 ↗First posted 18 May 2023cs.LG cond-mat.dis-nn stat.MLcited by 19

Not on the map. Why?

Abstract (arXiv)

We address the problem of denoising data from a Gaussian mixture using a two-layer non-linear autoencoder with tied weights and a skip connection. We consider the high-dimensional limit where the number of training samples and the input dimension jointly tend to infinity while the number of hidden units remains bounded. We provide closed-form expressions for the denoising mean-squared test error. Building on this result, we quantitatively characterize the advantage of the considered architecture over the autoencoder without the skip connection that relates closely to principal component analysis. We further show that our results accurately capture the learning curves on a range of real data sets.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 32 of its 49 references that are in the map. First posted on arXiv 18 May 2023.

Builds on

32 of 49 references.

References · 32

2014Adam: A Method for Stochastic Optimizationcited by 36,960
2020Denoising Diffusion Probabilistic Modelscited by 20,851
2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
2015Accurate Image Super-Resolution Using Very Deep Convolutional Networkscited by 902
2015Deeply-Recursive Convolutional Network for Image Super-Resolutioncited by 317
2020Spectrum Dependent Learning Curves in Kernel Regression and Wide Neural Networkscited by 200
2015Statistical physics of inference: Thresholds and algorithmscited by 128
2020Generalisation error in learning with random features and the hidden manifold modelcited by 126
2021Learning curves of generic features maps for realistic datasets with a teacher-student modelcited by 126
2019Implicit Regularization of Discrete Gradient Dynamics in Linear Neural Networkscited by 122

20 more

2017Optimal Errors and Phase Transitions in High-Dimensional Generalized Linear Modelscited by 115
2020The Gaussian equivalence of generative models for learning with shallow neural networkscited by 113
2022Universality of empirical risk minimizationcited by 73
2018The committee machine: Computational to statistical gaps in learning a two-layers neural networkcited by 58
2021Learning Gaussian Mixtures with Generalised Linear Models: Precise Asymptotics in High-dimensionscited by 52
2020Generalization error in high-dimensional perceptrons: Approaching Bayes error with convex optimizationcited by 51
2019Loss Landscapes of Regularized Linear Autoencoderscited by 45
2019Overparameterized Neural Networks Implement Associative Memorycited by 45
2023Bayes-optimal Learning of Deep Random Networks of Extensive-widthcited by 41
2016Image Restoration Using Convolutional Auto-encoders with Symmetric Skip Connectionscited by 30
2022The dynamics of representation learning in shallow, non-linear autoencoderscited by 27
2020Regularized linear autoencoders recover the principal components, eventuallycited by 26
2022Contrasting random and learned features in deep Bayesian linear regressioncited by 24
2022Error Scaling Laws for Kernel Classification under Source and Capacity Conditionscited by 17
2018Learning Dynamics of Linear Denoising Autoencoderscited by 16
2019Mean-field inference methods for neural networkscited by 16
2019Benefits of Jointly Training Autoencoders: An Improved Neural Tangent Kernel Analysiscited by 13
2021Analysis of feature learning in weight-tied autoencoders via the mean field lenscited by 13
2021A Mechanism for Producing Aligned Latent Spaces with Autoencoderscited by 5
2022Fundamental Limits of Two-layer Autoencoders, and Achieving Them with Gradient Methodscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 7; 2025: 4; 2026: 6.

19 papers.

Sort:

By area:

2024An analytic theory of creativity in convolutional diffusion modelscited by 103
2023Analysis of learning a flow-based generative model from limited sample complexitycited by 38
2024A phase transition between positional and semantic learning in a solvable model of dot-product attentioncited by 30
2025A solvable model of learning generative diffusion: theory and insightscited by 14
2024High-dimensional learning of narrow neural networkscited by 12
2024On Regularization via Early Stopping for Least Squares Regressioncited by 10
2025Generalization of Diffusion Models Arises with a Balanced Representation Spacecited by 10
2026A solvable high-dimensional model where nonlinear autoencoders learn structure invisible to PCA while test loss misaligns with generalizationcited by 7
2023High-dimensional Asymptotics of VAEs: Threshold of Posterior Collapse and Dataset-Size Dependence of Rate-Distortion Curvecited by 5
2026A theory of learning data statistics in diffusion models, from easy to hardcited by 5
2024Compression of Structured Data with Autoencoders: Provable Benefit of Nonlinearities and Depthcited by 4
2025A statistical physics framework for optimal learningcited by 3

7 more

2024KAE: Kolmogorov-Arnold Auto-Encoder for Representation Learningcited by 2
2024Statistical Mechanics of Min-Max Problemscited by 1
2026Multi-Subspace Multi-Modal Modeling for Diffusion Models: Estimation, Convergence and Mixture of Expertscited by 1
2026Memorisation, convergence and generalisation in generative modelscited by 1
2025Impact of Bottleneck Layers and Skip Connections on the Generalization of Linear Denoising Autoencoderscited by 0
2026Provably Learning Diffusion Models under the Manifold Hypothesis: Collapse and Refinecited by 0
2026A prism hierarchy of learning regimes in large linear autoencoderscited by 0