mlsanityData to 14 Oct 2026

Loss Landscapes of Regularized Linear Autoencoders

arXiv 1901.08168 ↗First posted 23 Jan 2019cs.LG stat.MLcited by 46

Not on the map. Why?

Abstract (arXiv)

Autoencoders are a deep learning model for representation learning. When trained to minimize the distance between the data and its reconstruction, linear autoencoders (LAEs) learn the subspace spanned by the top principal directions but cannot learn the principal directions themselves. In this paper, we prove that L2-regularized LAEs are symmetric at all critical points and learn the principal directions as the left singular vectors of the decoder. We smoothly parameterize the critical manifold and relate the minima to the MAP estimate of probabilistic PCA. We illustrate these results empirically and consider implications for PCA algorithms, computational neuroscience, and the algebraic topology of learning.

Key facts. Cited by 46 papers in the map (data through 14 Oct 2026). First posted on arXiv 23 Jan 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 8; 2022: 7; 2023: 9; 2024: 5; 2025: 8; 2026: 9.

46 papers.

Sort:

By area:

  1. 2021A Geometric Analysis of Neural Collapse with Unconstrained Featurescited by 227
  2. 2021Noether's Learning Dynamics: Role of Symmetry Breaking in Neural Networkscited by 50
  3. 2023Stochastic Collapse: How Gradient Noise Attracts SGD Dynamics Towards Simpler Subnetworkscited by 39
  4. 2022Disentanglement with Biological Constraints: A Theory of Functional Cell Typescited by 30
  5. 2021Analyzing Monotonic Linear Interpolation in Neural Network Loss Landscapescited by 28
  6. 2022The dynamics of representation learning in shallow, non-linear autoencoderscited by 27
  7. 2023An SDE for Modeling SAM: Theory and Insightscited by 22
  8. 2023High-dimensional Asymptotics of Denoising Autoencoderscited by 19
  9. 2025Understanding Representation Dynamics of Diffusion Models via Low-Dimensional Modelingcited by 18
  10. 2021Analysis of feature learning in weight-tied autoencoders via the mean field lenscited by 14
  11. 2021Deep Bayesian inference for seismic imaging with taskscited by 12
  12. 2025Identifiability of Deep Polynomial Neural Networkscited by 12
34 more
  1. 2021Anytime Sampling for Autoregressive Models via Ordered Autoencodingcited by 11
  2. 2023Non-Parametric Representation Learning with Kernelscited by 11
  3. 2025Generalization of Diffusion Models Arises with a Balanced Representation Spacecited by 11
  4. 2025Learning on a Razor's Edge: Identifiability and Singularity of Polynomial Neural Networkscited by 9
  5. 2024Stabilizing Sharpness-aware Minimization Through A Simple Renormalization Strategycited by 8
  6. 2023Stochastic Gradient Descent-Induced Drift of Representation in a Two-Layer Neural Networkcited by 6
  7. 2021How and When Random Feedback Works: A Case Study of Low-Rank Matrix Factorizationcited by 5
  8. 2024Compression of Structured Data with Autoencoders: Provable Benefit of Nonlinearities and Depthcited by 5
  9. 2023Neural Image Compression: Generalization, Robustness, and Spectral Biasescited by 4
  10. 2025Posterior Collapse as a Phase Transition in Variational Autoencoderscited by 4
  11. 2026Subspace-Aware Sparse Autoencoders for Effective Mechanistic Interpretabilitycited by 4
  12. 2022Fundamental Limits of Two-layer Autoencoders, and Achieving Them with Gradient Methodscited by 3
  13. 2023Representation Learning Dynamics of Self-Supervised Modelscited by 3
  14. 2024The Persian Rug: solving toy models of superposition using large-scale symmetriescited by 3
  15. 2022Memorization-Dilation: Modeling Neural Collapse Under Label Noisecited by 2
  16. 2022Evaluating Disentanglement in Generative Models Without Knowledge of Latent Factorscited by 2
  17. 2023The Effect of SGD Batch Size on Autoencoder Learning: Sparsity, Sharpness, and Feature Learningcited by 2
  18. 2023Brain-like Flexible Visual Inference by Harnessing Feedback-Feedforward Alignmentcited by 2
  19. 2025A Complete Loss Landscape Analysis of Regularized Deep Matrix Factorizationcited by 2
  20. 2026PCAE: Learning Ordered Representations in Latent Space for Intrinsic Dimension Estimation via Principal Component Autoencodercited by 2
  21. 2021Principal Bit Analysis: Autoencoding with Schur-Concave Losscited by 1
  22. 2022Improved Representation Learning Through Tensorized Autoencoderscited by 1
  23. 2026Stimulus symmetries can confound representational similarity analysescited by 1
  24. 2022The learning phases in NN: From Fitting the Majority to Fitting a Fewcited by 0
  25. 2024Manifold Metric: A Loss Landscape Approach for Predicting Model Performancecited by 0
  26. 2024Beta-Sigma VAE: Separating beta and decoder variance in Gaussian variational autoencodercited by 0
  27. 2025Theoretical Foundations of Representation Learning using Unlabeled Data: Statistics and Optimizationcited by 0
  28. 2025RAE: A Neural Network Dimensionality Reduction Method for Nearest Neighbors Preservation in Vector Searchcited by 0
  29. 2026Objective-Specific Privileged Bases via Full-Prefix Matryoshka Learningcited by 0
  30. 2026Singular Learning and Occam's Razor in Deep Monomial Networkscited by 0
  31. 2026When Does Deep Representation Learning Help Single-Cell Clustering? A Sensitivity-Aware Diagnostic Benchmark for Biomedical AI Pipelinescited by 0
  32. 2026On the Regularization Landscape for the Linear Recommendation Modelscited by 0
  33. 2026Sharp Reconstruction Bounds for Autoencoders Using the Same Forward Mapcited by 0
  34. 2026Broken scale symmetries in undercomplete linear autoencoderscited by 0