# Unsupervised Learning of Debiased Representations with Pseudo-Attributes

arXiv: https://arxiv.org/abs/2108.02943 · HTML: https://mlsanity.com/paper/2108.02943/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 41 references that are in the map.
- First posted on arXiv 6 Aug 2021.

## Builds on (most cited first, 29 of 29)

- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning](https://mlsanity.com/paper/1612.06890/) (2016)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization](https://mlsanity.com/paper/1911.08731/) (2019)
- [Deeper, Broader and Artier Domain Generalization](https://mlsanity.com/paper/1710.03077/) (2017)
- [Agnostic Federated Learning](https://mlsanity.com/paper/1902.00146/) (2019)
- [Domain Generalization by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1903.06864/) (2019)
- [Fair Resource Allocation in Federated Learning](https://mlsanity.com/paper/1905.10497/) (2019)
- [An Investigation of Why Overparameterization Exacerbates Spurious Correlations](https://mlsanity.com/paper/2005.04345/) (2020)
- [Don't Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biases](https://mlsanity.com/paper/1909.03683/) (2019)
- [Women also Snowboard: Overcoming Bias in Captioning Models](https://mlsanity.com/paper/1803.09797/) (2018)
- [Empirical Risk Minimization under Fairness Constraints](https://mlsanity.com/paper/1802.08626/) (2018)
- [No Subclass Left Behind: Fine-Grained Robustness in Coarse-Grained Classification Problems](https://mlsanity.com/paper/2011.12945/) (2020)
- [Learning De-biased Representations with Biased Representations](https://mlsanity.com/paper/1910.02806/) (2019)
- [RUBi: Reducing Unimodal Biases in Visual Question Answering](https://mlsanity.com/paper/1906.10169/) (2019)
- [Statistics of Robust Optimization: A Generalized Empirical Likelihood Approach](https://mlsanity.com/paper/1610.03425/) (2016)
- [Learning to Optimize Domain Specific Normalization for Domain Generalization](https://mlsanity.com/paper/1907.04275/) (2019)
- [REPAIR: Removing Representation Bias by Dataset Resampling](https://mlsanity.com/paper/1904.07911/) (2019)
- [The Frontiers of Fairness in Machine Learning](https://mlsanity.com/paper/1810.08810/) (2018)
- [REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasets](https://mlsanity.com/paper/2004.07999/) (2020)
- [Learning from Failure: Training Debiased Classifier from Biased Classifier](https://mlsanity.com/paper/2007.02561/) (2020)
- [Learning Robust Representations by Projecting Superficial Statistics Out](https://mlsanity.com/paper/1903.06256/) (2019)
- [Distributionally Robust Bayesian Optimization](https://mlsanity.com/paper/2002.09038/) (2020)
- [Distributionally Robust Losses for Latent Covariate Mixtures](https://mlsanity.com/paper/2007.13982/) (2020)
- [Towards Accuracy-Fairness Paradox: Adversarial Example-based Data Augmentation for Visual Debiasing](https://mlsanity.com/paper/2007.13632/) (2020)
- [Wasserstein Distributionally Robust Optimization and Variation Regularization](https://mlsanity.com/paper/1712.06050/) (2017)
- [Racial Faces in-the-Wild: Reducing Racial Bias by Information Maximization Adaptation Network](https://mlsanity.com/paper/1812.00194/) (2018)

## Built on by (most cited first, 30 of 44)

- [A Whac-A-Mole Dilemma: Shortcuts Come in Multiples Where Mitigating One Amplifies Others](https://mlsanity.com/paper/2212.04825/) (2022)
- [Fast Model Debias with Machine Unlearning](https://mlsanity.com/paper/2310.12560/) (2023)
- [Discover and Cure: Concept-aware Mitigation of Spurious Correlation](https://mlsanity.com/paper/2305.00650/) (2023)
- [Avoiding spurious correlations via logit correction](https://mlsanity.com/paper/2212.01433/) (2022)
- [Seeing is not Believing: Robust Reinforcement Learning against Spurious Correlation](https://mlsanity.com/paper/2307.07907/) (2023)
- [Mitigating Dataset Bias by Using Per-sample Gradient](https://mlsanity.com/paper/2205.15704/) (2022)
- [Learning Debiased and Disentangled Representations for Semantic Segmentation](https://mlsanity.com/paper/2111.00531/) (2021)
- [Information-Theoretic Bias Reduction via Causal View of Spurious Correlation](https://mlsanity.com/paper/2201.03121/) (2022)
- [Fairness and Bias Mitigation in Computer Vision: A Survey](https://mlsanity.com/paper/2408.02464/) (2024)
- [Ferrari: Federated Feature Unlearning via Optimizing Feature Sensitivity](https://mlsanity.com/paper/2405.17462/) (2024)
- [Fighting Fire with Fire: Contrastive Debiasing without Bias-free Data via Generative Bias-transformation](https://mlsanity.com/paper/2112.01021/) (2021)
- [Bitrate-Constrained DRO: Beyond Worst Case Robustness To Unknown Group Shifts](https://mlsanity.com/paper/2302.02931/) (2023)
- [Out of Spuriousity: Improving Robustness to Spurious Correlations without Group Annotations](https://mlsanity.com/paper/2407.14974/) (2024)
- [Prompt-driven Latent Domain Generalization for Medical Image Classification](https://mlsanity.com/paper/2401.03002/) (2024)
- [ExMap: Leveraging Explainability Heatmaps for Unsupervised Group Robustness to Spurious Correlations](https://mlsanity.com/paper/2403.13870/) (2024)
- [Mitigating Spurious Correlation via Distributionally Robust Learning with Hierarchical Ambiguity Sets](https://mlsanity.com/paper/2510.02818/) (2025)
- [Bias Amplification Enhances Minority Group Performance](https://mlsanity.com/paper/2309.06717/) (2023)
- [DeNetDM: Debiasing by Network Depth Modulation](https://mlsanity.com/paper/2403.19863/) (2024)
- [Diffexplainer: Towards Cross-modal Global Explanations with Diffusion Models](https://mlsanity.com/paper/2404.02618/) (2024)
- [A survey of Identification and mitigation of Machine Learning algorithmic biases in Image Analysis](https://mlsanity.com/paper/2210.04491/) (2022)
- [A Simple Remedy for Dataset Bias via Self-Influence: A Mislabeled Sample Perspective](https://mlsanity.com/paper/2411.00360/) (2024)
- [Attention IoU: Examining Biases in CelebA using Attention Maps](https://mlsanity.com/paper/2503.19846/) (2025)
- [Echoes: Unsupervised Debiasing via Pseudo-bias Labeling in an Echo Chamber](https://mlsanity.com/paper/2305.04043/) (2023)
- [Language-guided Detection and Mitigation of Unknown Dataset Bias](https://mlsanity.com/paper/2406.02889/) (2024)
- [Looking at Model Debiasing through the Lens of Anomaly Detection](https://mlsanity.com/paper/2407.17449/) (2024)
- [Mitigating Spurious Correlations via Disagreement Probability](https://mlsanity.com/paper/2411.01757/) (2024)
- [NeuronTune: Towards Self-Guided Spurious Bias Mitigation](https://mlsanity.com/paper/2505.24048/) (2025)
- [Debugging Concept Bottleneck Models through Removal and Retraining](https://mlsanity.com/paper/2509.21385/) (2025)
- [Language-assisted Vision Model Debugger: A Sample-Free Approach to Finding and Fixing Bugs](https://mlsanity.com/paper/2312.05588/) (2023)
- [Common-Sense Bias Modeling for Classification Tasks](https://mlsanity.com/paper/2401.13213/) (2024)

## Abstract (arXiv)

Dataset bias is a critical challenge in machine learning since it often leads to a negative impact on a model due to the unintended decision rules captured by spurious correlations. Although existing works often handle this issue based on human supervision, the availability of the proper annotations is impractical and even unrealistic. To better tackle the limitation, we propose a simple but effective unsupervised debiasing technique. Specifically, we first identify pseudo-attributes based on the results from clustering performed in the feature embedding space even without an explicit bias attribute supervision. Then, we employ a novel cluster-wise reweighting scheme to learn debiased representation; the proposed method prevents minority groups from being discounted for minimizing the overall loss, which is desirable for worst-case generalization. The extensive experiments demonstrate the outstanding performance of our approach on multiple standard benchmarks, even achieving the competitive accuracy to the supervised counterpart.