An Investigation of Representation and Allocation Harms in Contrastive Learning
Not on the map. Why?
Abstract (arXiv)
The effect of underrepresentation on the performance of minority groups is known to be a serious problem in supervised learning settings; however, it has been underexplored so far in the context of self-supervised learning (SSL). In this paper, we demonstrate that contrastive learning (CL), a popular variant of SSL, tends to collapse representations of minority groups with certain majority groups. We refer to this phenomenon as representation harm and demonstrate it on image and text datasets using the corresponding popular CL methods. Furthermore, our causal mediation analysis of allocation harm on a downstream classification task reveals that representation harm is partly responsible for it, thus emphasizing the importance of studying and mitigating representation harm. Finally, we provide a theoretical explanation for representation harm using a stochastic block model that leads to a representational neural collapse in a contrastive learning setting.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 50 of its 63 references that are in the map. First posted on arXiv 2 Oct 2023.
Builds on
50 of 63 references.
References · 50
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
- 2020Exploring Simple Siamese Representation Learningcited by 2,853
- 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
- 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
- 2016Equality of Opportunity in Supervised Learningcited by 2,000
- 2016node2vec: Scalable Feature Learning for Networkscited by 1,913
- 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,393
- 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 806
- 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 713
38 more
- 2019Self-supervised Visual Feature Learning with Deep Neural Networks: A Surveycited by 656
- 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 589
- 2018A Reductions Approach to Fair Classificationcited by 585
- 2019A Theoretical Analysis of Contrastive Unsupervised Representation Learningcited by 530
- 2021Just Train Twice: Improving Group Robustness without Training Group Informationcited by 522
- 2018Fairness Without Demographics in Repeated Loss Minimizationcited by 363
- 2019Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Settingcited by 346
- 2020Keystroke Dynamics as Part of Lifeloggingcited by 334
- 2021Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Losscited by 311
- 2020An Investigation of Why Overparameterization Exacerbates Spurious Correlationscited by 310
- 2021Graph Self-Supervised Learning: A Surveycited by 289
- 2013Direct and Indirect Effectscited by 276
- 2021Understanding self-supervised Learning Dynamics without Contrastive Pairscited by 254
- 2018What is the Effect of Importance Weighting in Deep Learning?cited by 250
- 2021A Geometric Analysis of Neural Collapse with Unconstrained Featurescited by 226
- 2020TERA: Self-Supervised Learning of Transformer Encoder Representation for Speechcited by 186
- 2021Simple data balancing achieves competitive worst-group-accuracycited by 177
- 2020CERT: Contrastive Self-supervised Learning for Language Understandingcited by 169
- 2021Exploring Deep Neural Networks via Layer-Peeled Model: Minority Collapse in Imbalanced Trainingcited by 166
- 2021Neural Collapse Under MSE Loss: Proximity to and Dynamics on the Central Pathcited by 156
- 2020Predicting What You Already Know Helps: Provable Self-Supervised Learningcited by 152
- 2020Contrastive learning, multi-view redundancy, and linear modelscited by 146
- 2020Neural collapse with unconstrained featurescited by 140
- 2023Social Biases through the Text-to-Image Generation Lenscited by 126
- 2021Self-supervised Learning is More Robust to Dataset Imbalancecited by 122
- 2021On the Role of Neural Collapse in Transfer Learningcited by 100
- 2022Self-Supervised Learning for Recommender Systems: A Surveycited by 95
- 2022Self-Supervised Learning for Videos: A Surveycited by 85
- 2022Limitations of Neural Collapse for Understanding Generalization in Deep Learningcited by 63
- 2017Deep Over-sampling Framework for Classifying Imbalanced Datacited by 58
- 2022The Hidden Uniform Cluster Prior in Self-Supervised Learningcited by 57
- 2020Posterior Re-calibration for Imbalanced Datasetscited by 53
- 2020Neural Collapse with Cross-Entropy Losscited by 52
- 2021Large-Scale Self- and Semi-Supervised Learning for Speech Translationcited by 36
- 2019Fairness in representation: quantifying stereotyping as a representational harmcited by 26
- 2022Contrastive Learning with Boosted Memorizationcited by 26
- 2021Conditional Contrastive Learning for Improving Fairness in Self-Supervised Learningcited by 25
- 2022Outlier-Robust Group Inference via Gradient Space Clusteringcited by 2
Built on by
Citing papers per year: 2024: 1; 2025: 1.
2 papers.