mlsanityData to 7 Oct 2026

An Empirical Study on Disentanglement of Negative-free Contrastive Learning

arXiv 2206.04756 ↗First posted 9 Jun 2022cs.LG cs.AI cs.CVcited by 12

Not on the map. Why?

Abstract (arXiv)

Negative-free contrastive learning methods have attracted a lot of attention with simplicity and impressive performances for large-scale pretraining. However, its disentanglement property remains unexplored. In this paper, we examine negative-free contrastive learning methods to study the disentanglement property empirically. We find that existing disentanglement metrics fail to make meaningful measurements for high-dimensional representation models, so we propose a new disentanglement metric based on Mutual Information between latent representations and data factors. With this proposed metric, we benchmark the disentanglement property of negative-free contrastive learning on both popular synthetic datasets and a real-world dataset CelebA. Our study shows that the investigated methods can learn a well-disentangled subset of representation. As far as we know, we are the first to extend the study of disentangled representation learning to high-dimensional representation space and introduce negative-free contrastive learning methods into this area. The source code of this paper is available at urlhttps://github.com/noahcao/disentanglementlibmed.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 39 of its 60 references that are in the map. First posted on arXiv 9 Jun 2022.

Builds on

39 of 60 references.

References · 39

  1. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  2. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  3. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  4. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  5. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
  6. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
  7. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,393
  8. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 929
  9. 2016InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Netscited by 858
  10. 2020What Makes for Good Views for Contrastive Learning?cited by 843
  11. 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 703
  12. 2018Disentangling by Factorisingcited by 614
27 more
  1. 2019A Theoretical Analysis of Contrastive Unsupervised Representation Learningcited by 530
  2. 2019Variational Autoencoders and Nonlinear ICA: A Unifying Frameworkcited by 494
  3. 2021Understanding Dimensional Collapse in Contrastive Self-supervised Learningcited by 374
  4. 2014Discriminative Unsupervised Feature Learning with Exemplar Convolutional Neural Networkscited by 299
  5. 2018Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learningcited by 278
  6. 2016Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICAcited by 270
  7. 2017Variational Inference of Disentangled Latent Concepts from Unlabeled Observationscited by 240
  8. 2020Weakly-Supervised Disentanglement Without Compromisescited by 237
  9. 2021Contrastive Learning Inverts the Data Generating Processcited by 220
  10. 2017Improved Texture Networks: Maximizing Quality and Diversity in Feed-forward Stylization and Texture Synthesiscited by 215
  11. 2021On Feature Decorrelation in Self-Supervised Learningcited by 160
  12. 2020Predicting What You Already Know Helps: Provable Self-Supervised Learningcited by 152
  13. 2020Contrastive learning, multi-view redundancy, and linear modelscited by 146
  14. 2020Demystifying Contrastive Self-Supervised Learning: Invariances, Augmentations and Dataset Biasescited by 143
  15. 2018Recent Advances in Autoencoder-Based Representation Learningcited by 130
  16. 2015Deep Convolutional Inverse Graphics Networkcited by 121
  17. 2020Towards Nonlinear Disentanglement in Natural Data with Temporal Sparse Codingcited by 121
  18. 2018Unsupervised Learning of Object Landmarks through Conditional Image Generationcited by 113
  19. 2020What makes instance discrimination good for transfer learning?cited by 111
  20. 2020ICE-BeeM: Identifiable Conditional Energy-Based Deep Models Based on Nonlinear ICAcited by 102
  21. 2018Learning Deep Disentangled Embeddings with the F-Statistic Losscited by 101
  22. 2019Are Disentangled Representations Helpful for Abstract Visual Reasoning?cited by 100
  23. 2020BYOL works even without batch statisticscited by 95
  24. 2021Disentangling Identifiable Features from Noisy Data with Structured Nonlinear ICAcited by 64
  25. 2020Understanding Self-supervised Learning with Dual Deep Networkscited by 62
  26. 2018Life-Long Disentangled Representation Learning with Cross-Domain Latent Homologiescited by 43
  27. 2020Self-supervised Learning from a Multi-view Perspectivecited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 5; 2025: 2; 2026: 1.

12 papers.

  1. 2023DisDiff: Unsupervised Disentanglement of Diffusion Probabilistic Modelscited by 30
  2. 2024Data Augmentation of Contrastive Learning is Estimating Positive-incentive Noisecited by 25
  3. 2023Towards the Sparseness of Projection Head in Self-Supervised Learningcited by 4
  4. 2024α-TCVAE: On the relationship between Disentanglement and Diversitycited by 4
  5. 2023Correcting Flaws in Common Disentanglement Metricscited by 2
  6. 2024CFASL: Composite Factor-Aligned Symmetry Learning for Disentanglement in Variational AutoEncodercited by 2
  7. 2023Variantional autoencoder with decremental information bottleneck for disentanglementcited by 1
  8. 2024Revisiting Disentanglement in Downstream Tasks: A Study on Its Necessity for Abstract Visual Reasoningcited by 1
  9. 2024Transferring disentangled representations: bridging the gap between synthetic and real imagescited by 1
  10. 2025scSSL-Bench: Benchmarking Self-Supervised Learning for Single-Cell Datacited by 1
  11. 2025Disentangled representations of microscopy imagescited by 0
  12. 2026XFACTORS: Disentangled Information Bottleneck via Contrastive Supervisioncited by 0