# FACTS: First Amplify Correlations and Then Slice to Discover Bias

arXiv: https://arxiv.org/abs/2309.17430 · HTML: https://mlsanity.com/paper/2309.17430/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 43 references that are in the map.
- First posted on arXiv 29 Sep 2023.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [Semantics derived automatically from language corpora contain human-like biases](https://mlsanity.com/paper/1608.07187/) (2016)
- [ClipCap: CLIP Prefix for Image Captioning](https://mlsanity.com/paper/2111.09734/) (2021)
- [Just Train Twice: Improving Group Robustness without Training Group Information](https://mlsanity.com/paper/2107.09044/) (2021)
- [Unsupervised Embedding Learning via Invariant and Spreading Instance Feature](https://mlsanity.com/paper/1904.03436/) (2019)
- [Hidden Stratification Causes Clinically Meaningful Failures in Machine Learning for Medical Imaging](https://mlsanity.com/paper/1909.12475/) (2019)
- [Domino: Discovering Systematic Errors with Cross-Modal Embeddings](https://mlsanity.com/paper/2203.14960/) (2022)
- [Salient ImageNet: How to discover spurious features in Deep Learning?](https://mlsanity.com/paper/2110.04301/) (2021)
- [Spread Spurious Attribute: Improving Worst-group Accuracy with Spurious Attribute Estimation](https://mlsanity.com/paper/2204.02070/) (2022)
- [NICO++: Towards Better Benchmarking for Domain Generalization](https://mlsanity.com/paper/2204.08040/) (2022)
- [Leveraging Sparse Linear Layers for Debuggable Deep Networks](https://mlsanity.com/paper/2105.04857/) (2021)
- [Distilling Model Failures as Directions in Latent Space](https://mlsanity.com/paper/2206.14754/) (2022)
- [Predictive Inequity in Object Detection](https://mlsanity.com/paper/1902.11097/) (2019)
- [Diagnosing and Rectifying Vision Models using Language](https://mlsanity.com/paper/2302.04269/) (2023)
- [Understanding Failures of Deep Networks via Robust Feature Extraction](https://mlsanity.com/paper/2012.01750/) (2020)
- [Deep Learning Predicts Hip Fracture using Confounding Patient and Healthcare Variables](https://mlsanity.com/paper/1811.03695/) (2018)
- [Model Agnostic Sample Reweighting for Out-of-Distribution Learning](https://mlsanity.com/paper/2301.09819/) (2023)
- [ModelDiff: A Framework for Comparing Learning Algorithms](https://mlsanity.com/paper/2211.12491/) (2022)
- [Deep Learning Applied to Chest X-Rays: Exploiting and Preventing Shortcuts](https://mlsanity.com/paper/2009.10132/) (2020)
- [UDIS: Unsupervised Discovery of Bias in Deep Visual Recognition Models](https://mlsanity.com/paper/2110.15499/) (2021)
- [Measures of Information Reflect Memorization Patterns](https://mlsanity.com/paper/2210.09404/) (2022)
- [AGRO: Adversarial Discovery of Error-prone groups for Robust Optimization](https://mlsanity.com/paper/2212.00921/) (2022)

## Built on by (most cited first, 26 of 26)

- [The Clever Hans Mirage: A Comprehensive Survey on Spurious Correlations in Machine Learning](https://mlsanity.com/paper/2402.12715/) (2024)
- [Integrating LLMs with ITS: Recent Advances, Potentials, Challenges, and Future Directions](https://mlsanity.com/paper/2501.04437/) (2025)
- [Fairness and Bias Mitigation in Computer Vision: A Survey](https://mlsanity.com/paper/2408.02464/) (2024)
- [HiBug2: Efficient and Interpretable Error Slice Discovery for Comprehensive Model Debugging](https://mlsanity.com/paper/2501.16751/) (2025)
- [Efficient Bias Mitigation Without Privileged Information](https://mlsanity.com/paper/2409.17691/) (2024)
- [On Large Multimodal Models as Open-World Image Classifiers](https://mlsanity.com/paper/2503.21851/) (2025)
- [LADDER: Language-Driven Slice Discovery and Error Rectification in Vision Classifiers](https://mlsanity.com/paper/2408.07832/) (2024)
- [Visual Data Diagnosis and Debiasing with Concept Graphs](https://mlsanity.com/paper/2409.18055/) (2024)
- [What could go wrong? Discovering and describing failure modes in computer vision](https://mlsanity.com/paper/2408.04471/) (2024)
- [ConceptScope: Characterizing Dataset Bias via Disentangled Visual Concepts](https://mlsanity.com/paper/2510.26186/) (2025)
- [Multi-View Incongruity Learning for Multimodal Sarcasm Detection](https://mlsanity.com/paper/2412.00756/) (2024)
- [The Impact of Coreset Selection on Spurious Correlations and Group Robustness](https://mlsanity.com/paper/2507.11690/) (2025)
- [Subgroup Performance Analysis in Hidden Stratifications](https://mlsanity.com/paper/2503.10382/) (2025)
- [ViG-Bias: Visually Grounded Bias Discovery and Mitigation](https://mlsanity.com/paper/2407.01996/) (2024)
- [Superclass-Guided Representation Disentanglement for Spurious Correlation Mitigation](https://mlsanity.com/paper/2508.08570/) (2025)
- [FailureAtlas:Mapping the Failure Landscape of T2I Models via Active Exploration](https://mlsanity.com/paper/2509.21995/) (2025)
- [Does Weak-to-strong Generalization Happen under Spurious Correlations?](https://mlsanity.com/paper/2509.24005/) (2025)
- [Language-assisted Vision Model Debugger: A Sample-Free Approach to Finding and Fixing Bugs](https://mlsanity.com/paper/2312.05588/) (2023)
- [Distributionally robust self-supervised learning for tabular data](https://mlsanity.com/paper/2410.08511/) (2024)
- [Let Samples Speak: Mitigating Spurious Correlation by Exploiting the Clusterness of Samples](https://mlsanity.com/paper/2512.22874/) (2025)
- [Towards Real-world Debiasing: Rethinking Evaluation, Challenge, and Solution](https://mlsanity.com/paper/2405.15240/) (2024)
- [Bias-Aware Machine Unlearning: Towards Fairer Vision Models via Controllable Forgetting](https://mlsanity.com/paper/2509.07456/) (2025)
- [LatentDiff: Scaling Semantic Dataset Comparison to Millions of Images](https://mlsanity.com/paper/2605.00899/) (2026)
- [CB-SLICE: Concept-Based Interpretable Error Slice Discovery](https://mlsanity.com/paper/2605.29836/) (2026)
- [Invisible Shortcuts: Why Vision Encoders Know Your Camera](https://mlsanity.com/paper/2608.05424/) (2026)
- [Beyond Benchmarks: Using VLMs to Reveal Systematic Classification Failures Under Real World Conditions](https://mlsanity.com/paper/2609.11126/) (2026)

## Abstract (arXiv)

Computer vision datasets frequently contain spurious correlations between task-relevant labels and (easy to learn) latent task-irrelevant attributes (e.g. context). Models trained on such datasets learn "shortcuts" and underperform on bias-conflicting slices of data where the correlation does not hold. In this work, we study the problem of identifying such slices to inform downstream bias mitigation strategies. We propose First Amplify Correlations and Then Slice to Discover Bias (FACTS), wherein we first amplify correlations to fit a simple bias-aligned hypothesis via strongly regularized empirical risk minimization. Next, we perform correlation-aware slicing via mixture modeling in bias-aligned feature space to discover underperforming data slices that capture distinct correlations. Despite its simplicity, our method considerably improves over prior work (by as much as 35% precision@10) in correlation bias identification across a range of diverse evaluation settings. Our code is available at: https://github.com/yvsriram/FACTS.