# SelecMix: Debiased Learning by Contradicting-pair Sampling

arXiv: https://arxiv.org/abs/2211.02291 · HTML: https://mlsanity.com/paper/2211.02291/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 52 references that are in the map.
- First posted on arXiv 4 Nov 2022.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels](https://mlsanity.com/paper/1805.07836/) (2018)
- [Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations](https://mlsanity.com/paper/1811.12359/) (2018)
- [Just Train Twice: Improving Group Robustness without Training Group Information](https://mlsanity.com/paper/2107.09044/) (2021)
- [The Pitfalls of Simplicity Bias in Neural Networks](https://mlsanity.com/paper/2006.07710/) (2020)
- [Puzzle Mix: Exploiting Saliency and Local Statistics for Optimal Mixup](https://mlsanity.com/paper/2009.06962/) (2020)
- [Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNet](https://mlsanity.com/paper/1904.00760/) (2019)
- [Learning Not to Learn: Training Deep Neural Networks with Biased Data](https://mlsanity.com/paper/1812.10352/) (2018)
- [Improving Out-of-Distribution Robustness via Selective Augmentation](https://mlsanity.com/paper/2201.00299/) (2022)
- [Learning De-biased Representations with Biased Representations](https://mlsanity.com/paper/1910.02806/) (2019)
- [How Does Mixup Help With Robustness and Generalization?](https://mlsanity.com/paper/2010.04819/) (2020)
- [RUBi: Reducing Unimodal Biases in Visual Question Answering](https://mlsanity.com/paper/1906.10169/) (2019)
- [REPAIR: Removing Representation Bias by Dataset Resampling](https://mlsanity.com/paper/1904.07911/) (2019)
- [Learning Robust Representations by Projecting Superficial Statistics Out](https://mlsanity.com/paper/1903.06256/) (2019)
- [Learning Debiased Representation via Disentangled Feature Augmentation](https://mlsanity.com/paper/2107.01372/) (2021)
- [EnD: Entangling and Disentangling deep representations for bias correction](https://mlsanity.com/paper/2103.02023/) (2021)
- [Learning from others' mistakes: Avoiding dataset biases without modeling them](https://mlsanity.com/paper/2012.01300/) (2020)
- [Evading the Simplicity Bias: Training a Diverse Set of Models Discovers Solutions with Superior OOD Generalization](https://mlsanity.com/paper/2105.05612/) (2021)
- [BiaSwap: Removing dataset bias with bias-tailored swapping augmentation](https://mlsanity.com/paper/2108.10008/) (2021)
- [Which Shortcut Cues Will DNNs Choose? A Study from the Parameter-Space Perspective](https://mlsanity.com/paper/2110.03095/) (2021)
- [Rich Feature Construction for the Optimization-Generalization Dilemma](https://mlsanity.com/paper/2203.15516/) (2022)
- [A Unified Analysis of Mixed Sample Data Augmentation: A Loss Function Perspective](https://mlsanity.com/paper/2208.09913/) (2022)
- [Learning Stable Classifiers by Transferring Unstable Features](https://mlsanity.com/paper/2106.07847/) (2021)

## Built on by (most cited first, 29 of 29)

- [Trustworthiness in Retrieval-Augmented Generation Systems: A Survey](https://mlsanity.com/paper/2409.10102/) (2024)
- [VisionTrap: Vision-Augmented Trajectory Prediction Guided by Textual Descriptions](https://mlsanity.com/paper/2407.12345/) (2024)
- [SYNAuG: Exploiting Synthetic Data for Data Imbalance Problems](https://mlsanity.com/paper/2308.00994/) (2023)
- [Navigate Beyond Shortcuts: Debiased Learning through the Lens of Neural Collapse](https://mlsanity.com/paper/2405.05587/) (2024)
- [Selective Mixup Helps with Distribution Shifts, But Not (Only) because of Mixup](https://mlsanity.com/paper/2305.16817/) (2023)
- [Bias Amplification Enhances Minority Group Performance](https://mlsanity.com/paper/2309.06717/) (2023)
- [Improving Robustness to Multiple Spurious Correlations by Multi-Objective Optimization](https://mlsanity.com/paper/2409.03303/) (2024)
- [Tailoring Mixup to Data for Calibration](https://mlsanity.com/paper/2311.01434/) (2023)
- [Enhancing Intrinsic Features for Debiasing via Investigating Class-Discerning Common Attributes in Bias-Contrastive Pair](https://mlsanity.com/paper/2404.19250/) (2024)
- [Mitigating Bias in Dataset Distillation](https://mlsanity.com/paper/2406.06609/) (2024)
- [Locality-aware Concept Bottleneck Model](https://mlsanity.com/paper/2508.14562/) (2025)
- [Partition-and-Debias: Agnostic Biases Mitigation via A Mixture of Biases-Specific Experts](https://mlsanity.com/paper/2308.10005/) (2023)
- [Slot-Mixup with Subsampling: A Simple Regularization for WSI Classification](https://mlsanity.com/paper/2311.17466/) (2023)
- [A Simple Remedy for Dataset Bias via Self-Influence: A Mislabeled Sample Perspective](https://mlsanity.com/paper/2411.00360/) (2024)
- [Diffusing DeBias: Synthetic Bias Amplification for Model Debiasing](https://mlsanity.com/paper/2502.09564/) (2025)
- [Selective Mixup Fine-Tuning for Optimizing Non-Decomposable Objectives](https://mlsanity.com/paper/2403.18301/) (2024)
- [Beyond Silence: Bias Analysis through Loss and Asymmetric Approach in Audio Anti-Spoofing](https://mlsanity.com/paper/2406.17246/) (2024)
- [Model Debiasing by Learnable Data Augmentation](https://mlsanity.com/paper/2408.04955/) (2024)
- [Brain Hematoma Marker Recognition Using Multitask Learning: SwinTransformer and Swin-Unet](https://mlsanity.com/paper/2505.06185/) (2025)
- [BiasEdit: A Training-Free Bias-Detect-and-Edit Framework for Learning Fair Visual Classifiers](https://mlsanity.com/paper/2605.28450/) (2026)
- [Revisiting the Dataset Bias Problem from a Statistical Perspective](https://mlsanity.com/paper/2402.03577/) (2024)
- [Towards Real-world Debiasing: Rethinking Evaluation, Challenge, and Solution](https://mlsanity.com/paper/2405.15240/) (2024)
- [Debiasify: Self-Distillation for Unsupervised Bias Mitigation](https://mlsanity.com/paper/2411.00711/) (2024)
- [A Critical Review of Predominant Bias in Neural Networks](https://mlsanity.com/paper/2502.11031/) (2025)
- [Towards Spatially Consistent Image Generation: On Incorporating Intrinsic Scene Properties into Diffusion Models](https://mlsanity.com/paper/2508.10382/) (2025)
- [GroupCoOp: Group-robust Fine-tuning via Group Prompt Learning](https://mlsanity.com/paper/2509.23781/) (2025)
- [BLADE: Bias-Linked Adaptive DEbiasing](https://mlsanity.com/paper/2510.04174/) (2025)
- [Bias In, Bias Out? Finding Unbiased Subnetworks in Vanilla Models](https://mlsanity.com/paper/2603.05582/) (2026)
- [Discovering Latent Groups for Robust Classification](https://mlsanity.com/paper/2606.23609/) (2026)

## Abstract (arXiv)

Neural networks trained with ERM (empirical risk minimization) sometimes learn unintended decision rules, in particular when their training data is biased, i.e., when training labels are strongly correlated with undesirable features. To prevent a network from learning such features, recent methods augment training data such that examples displaying spurious correlations (i.e., bias-aligned examples) become a minority, whereas the other, bias-conflicting examples become prevalent. However, these approaches are sometimes difficult to train and scale to real-world data because they rely on generative models or disentangled representations. We propose an alternative based on mixup, a popular augmentation that creates convex combinations of training examples. Our method, coined SelecMix, applies mixup to contradicting pairs of examples, defined as showing either (i) the same label but dissimilar biased features, or (ii) different labels but similar biased features. Identifying such pairs requires comparing examples with respect to unknown biased features. For this, we utilize an auxiliary contrastive model with the popular heuristic that biased features are learned preferentially during training. Experiments on standard benchmarks demonstrate the effectiveness of the method, in particular when label noise complicates the identification of bias-conflicting examples.