mlsanityData to 7 Oct 2026

Implicit Rank-Minimizing Autoencoder

arXiv 2010.00679 ↗First posted 1 Oct 2020cs.LG cs.CV stat.MLcited by 35

Not on the map. Why?

Abstract (arXiv)

An important component of autoencoders is the method by which the information capacity of the latent representation is minimized or limited. In this work, the rank of the covariance matrix of the codes is implicitly minimized by relying on the fact that gradient descent learning in multi-layer linear networks leads to minimum-rank solutions. By inserting a number of extra linear layers between the encoder and the decoder, the system spontaneously learns representations with a low effective dimension. The model, dubbed Implicit Rank-Minimizing Autoencoder (IRMAE), is simple, deterministic, and learns compact latent spaces. We demonstrate the validity of the method on several image generation and representation learning tasks.

Key facts. Cited by 35 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 7; 2022: 5; 2023: 9; 2024: 5; 2025: 3; 2026: 6.

35 papers.

Sort:

By area:

  1. 2021Understanding Dimensional Collapse in Contrastive Self-supervised Learningcited by 374
  2. 2021The Low-Rank Simplicity Bias in Deep Networkscited by 150
  3. 2022How Mask Matters: Towards Theoretical Understandings of Masked Autoencoderscited by 86
  4. 2021Data-driven discovery of intrinsic dynamicscited by 45
  5. 2023Disentanglement via Latent Quantizationcited by 40
  6. 2022Learning to Accelerate Partial Differential Equations via Latent Global Evolutioncited by 38
  7. 2021Self-Validation: Early Stopping for Single-Instance Deep Generative Priorscited by 15
  8. 2023Autoencoders for discovering manifold dimension and coordinates in data from complex dynamical systemscited by 13
  9. 2024Tripod: Three Complementary Inductive Biases for Disentangled Representation Learningcited by 13
  10. 2025Deep Weight Factorization: Sparse Learning Through the Lens of Artificial Symmetriescited by 13
  11. 2024Residual Connections Harm Generative Representation Learningcited by 10
  12. 2024Preventing Dimensional Collapse in Self-Supervised Learning via Orthogonality Regularizationcited by 9
23 more
  1. 2023Implicit Regularization for Group Sparsitycited by 8
  2. 2022Simpler is better: spectral regularization and up-sampling techniques for variational autoencoderscited by 6
  3. 2023Deep ReLU Networks Have Surprisingly Simple Polytopescited by 5
  4. 2024Rank Reduction Autoencoderscited by 5
  5. 2022Implicit Regularization with Polynomial Growth in Deep Tensor Factorizationcited by 3
  6. 2021Improved CNN-based Learning of Interpolation Filters for Low-Complexity Inter Prediction in Video Codingcited by 2
  7. 2023Building symmetries into data-driven manifold dynamics models for complex flows: application to two-dimensional Kolmogorov flowcited by 2
  8. 2026PCAE: Learning Ordered Representations in Latent Space for Intrinsic Dimension Estimation via Principal Component Autoencodercited by 2
  9. 2021Implicit Greedy Rank Learning in Autoencoders via Overparameterized Linear Networkscited by 1
  10. 2023Learning Low-Rank Latent Spaces with Simple Deterministic Autoencoder: Theoretical and Empirical Insightscited by 1
  11. 2024Lecture Notes on Linear Neural Networks: A Tale of Optimization and Generalization in Deep Learningcited by 1
  12. 2025Characterizing Continuous and Discrete Hybrid Latent Spaces for Structural Connectomescited by 1
  13. 2026TRIE: An Evaluation Framework for Stochastic PDE Surrogatescited by 1
  14. 2021Eccentric Regularization: Minimizing Hyperspherical Energy without explicit projectioncited by 0
  15. 2022A Study of Representational Properties of Unsupervised Anomaly Detection in Brain MRIcited by 0
  16. 2023Learning Linear Groups in Neural Networkscited by 0
  17. 2023A Theoretical and Empirical Study on the Convergence of Adam with an "Exact" Constant Step Size in Non-Convex Settingscited by 0
  18. 2023IHT-Inspired Neural Network for Single-Snapshot DOA Estimation with Sparse Linear Arrayscited by 0
  19. 2025Blending data and physics for reduced-order modeling of systems with spatiotemporal chaotic dynamicscited by 0
  20. 2026CCAR: Intrinsic Robustness as an Emergent Geometric Propertycited by 0
  21. 2026Partially Linear Autoencoders for Manifold Learning and Dimensionality Reductioncited by 0
  22. 2026A Theoretical Framework for Masked Pretraining (MPT)cited by 0
  23. 2026Manifold Projection and Iterative Autoencoder Refinement for Masked Language Modelingcited by 0