mlsanityData to 7 Oct 2026

Towards a Unified Framework of Contrastive Learning for Disentangled Representations

arXiv 2311.04774 ↗First posted 8 Nov 2023cs.LG stat.MLcited by 13

Not on the map. Why?

Abstract (arXiv)

Contrastive learning has recently emerged as a promising approach for learning data representations that discover and disentangle the explanatory factors of the data. Previous analyses of such approaches have largely focused on individual contrastive losses, such as noise-contrastive estimation (NCE) and InfoNCE, and rely on specific assumptions about the data generating process. This paper extends the theoretical guarantees for disentanglement to a broader family of contrastive methods, while also relaxing the assumptions about the data distribution. Specifically, we prove identifiability of the true latents for four contrastive losses studied in this paper, without imposing common independence assumptions. The theoretical findings are validated on several benchmark datasets. Finally, practical limitations of these methods are also investigated.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 38 of its 58 references that are in the map. First posted on arXiv 8 Nov 2023.

Builds on

38 of 58 references.

References · 38

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  4. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  5. 2020Shortcut Learning in Deep Neural Networkscited by 1,928
  6. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,393
  7. 2019Contrastive Multiview Codingcited by 1,297
  8. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,265
  9. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 929
  10. 2020What Makes for Good Views for Contrastive Learning?cited by 843
  11. 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 733
  12. 2019Variational Autoencoders and Nonlinear ICA: A Unifying Frameworkcited by 494
26 more
  1. 2020Keystroke Dynamics as Part of Lifeloggingcited by 334
  2. 2019On Mutual Information Maximization for Representation Learningcited by 315
  3. 2021Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Losscited by 311
  4. 2021Self-Supervised Learning with Data Augmentations Provably Isolates Content from Stylecited by 308
  5. 2018Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learningcited by 278
  6. 2016Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICAcited by 270
  7. 2020Weakly-Supervised Disentanglement Without Compromisescited by 237
  8. 2021Contrastive Learning Inverts the Data Generating Processcited by 220
  9. 2019Flexibly Fair Representation Learning by Disentanglementcited by 196
  10. 2016Exploring the Limits of Language Modelingcited by 172
  11. 2021Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICAcited by 158
  12. 2019On the Fairness of Disentangled Representationscited by 144
  13. 2022Weakly supervised causal representation learningcited by 140
  14. 2022CITRIS: Causal Identifiability from Temporal Intervened Sequencescited by 132
  15. 2020Intriguing Properties of Contrastive Lossescited by 129
  16. 2020Towards Nonlinear Disentanglement in Natural Data with Temporal Sparse Codingcited by 121
  17. 2018Noise Contrastive Estimation and Negative Sampling for Conditional Models: Consistency and Statistical Efficiencycited by 103
  18. 2020ICE-BeeM: Identifiable Conditional Energy-Based Deep Models Based on Nonlinear ICAcited by 102
  19. 2018Learning Deep Disentangled Embeddings with the F-Statistic Losscited by 101
  20. 2019Are Disentangled Representations Helpful for Abstract Visual Reasoning?cited by 100
  21. 2021Independent mechanism analysis, a new concept?cited by 96
  22. 2019The Incomplete Rosetta Stone Problem: Identifiability Results for Multi-View Nonlinear ICAcited by 86
  23. 2019Weakly Supervised Disentanglement with Guaranteescited by 82
  24. 2022Weakly Supervised Representation Learning with Sparse Perturbationscited by 64
  25. 2019Disentangling Factors of Variation Using Few Labelscited by 57
  26. 2021Learning disentangled representations via product manifold projectioncited by 20

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 5; 2026: 4.

13 papers.

Sort:

By area:

  1. 2024InfoNCE: Identifying the Gap Between Theory and Practicecited by 39
  2. 2024Self-supervised contrastive learning performs non-linear system identificationcited by 9
  3. 2024Causal Contrastive Learning for Counterfactual Regression Over Timecited by 7
  4. 2025SPACE: Noise Contrastive Estimation Stabilizes Self-Play Fine-Tuning for Large Language Modelscited by 7
  5. 2025ManiVID-3D: Generalizable View-Invariant Reinforcement Learning for Robotic Manipulation via Disentangled 3D Representationscited by 5
  6. 2024Disentangled Representation Learning with the Gromov-Monge Gapcited by 3
  7. 2025Mechanistic Independence: A Principle for Identifiable Disentangled Representationscited by 3
  8. 2025Provable Affine Identifiability of Nonlinear CCA under Latent Distributional Priorscited by 1
  9. 2025PLATONT: Learning a Platonic Representation for Unified Network Tomographycited by 0
  10. 2026Provable Subspace Identification of Nonlinear Multi-view CCAcited by 0
  11. 2026Chameleon: Style-Content Disentangled Framework for Cross-Domain Object Compositingcited by 0
  12. 2026Contrastive-Augmented Flow Matching for Style-Content Disentanglementcited by 0
1 more
  1. 2026Disentangling Multi-View Scanning in Mamba for Network Traffic Anomaly Detectioncited by 0