mlsanityData to 14 Oct 2026

Adversarial Dependence Minimization

arXiv 2502.03227 ↗First posted 5 Feb 2025cs.LG cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

Minimally redundant representations are typically learned by minimizing feature covariance. However, covariance-based methods fail to eliminate all dependencies/redundancies, as linearly uncorrelated variables can still exhibit nonlinear relationships. To address this, we introduce ADM, a differentiable algorithm that minimizes statistical dependence between feature dimensions through an adversarial game: auxiliary networks identify dependencies, while the encoder removes them. We prove that mutual independence is achieved at the global optimum, empirically verify convergence, and study three potential applications: extending PCA to nonlinear decorrelation, improving generalization in image classification, and preventing dimensional collapse in self-supervised learning. By promoting statistically independent representations, ADM paves the way for learning more robust, compressed, and generalizable representations across diverse applications.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 34 of its 89 references that are in the map. First posted on arXiv 5 Feb 2025.

Builds on

34 of 89 references.

References · 34

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  4. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  5. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  6. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  7. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  8. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,845
  9. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,838
  10. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,492
  11. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,457
  12. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 941
22 more
  1. 2016InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Netscited by 860
  2. 2018MINE: Mutual Information Neural Estimationcited by 683
  3. 2018Disentangling by Factorisingcited by 616
  4. 2017Large Batch Training of Convolutional Networkscited by 535
  5. 2019Variational Autoencoders and Nonlinear ICA: A Unifying Frameworkcited by 503
  6. 2021Understanding Dimensional Collapse in Contrastive Self-supervised Learningcited by 378
  7. 2020Whitening for Self-Supervised Representation Learningcited by 279
  8. 2016Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICAcited by 273
  9. 2021On Feature Decorrelation in Self-Supervised Learningcited by 160
  10. 2018Decorrelated Batch Normalizationcited by 83
  11. 2021Self-Supervised Learning with Kernel Dependence Maximizationcited by 77
  12. 2023To Compress or Not to Compress- Self-Supervised Learning and Information Theory: A Reviewcited by 72
  13. 2023No Representation Rules Them All in Category Discoverycited by 61
  14. 2022Function Classes for Identifiable Nonlinear Independent Component Analysiscited by 53
  15. 2017Learning Independent Features with Adversarial Nets for Non-linear ICAcited by 33
  16. 2022Contrastive Learning Can Find An Optimal Basis For Approximately View-Invariant Functionscited by 32
  17. 2023Understanding Augmentation-based Self-Supervised Representation Learning via RKHS Approximation and Regressioncited by 20
  18. 2020Learning Unbiased Representations via R'enyi Minimizationcited by 18
  19. 2023Neural Feature Learning in Function Spacecited by 12
  20. 2022Variance Covariance Regularization Enforces Pairwise Independence in Self-Supervised Representationscited by 10
  21. 2019Correspondence Analysis Using Neural Networkscited by 5
  22. 2024Beyond Pairwise Correlations: Higher-Order Redundancies in Self-Supervised Representation Learningcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Self-Supervised Learning with a Multi-Task Latent Space Objectivecited by 0