# Distilling Model Failures as Directions in Latent Space

arXiv: https://arxiv.org/abs/2206.14754 · HTML: https://mlsanity.com/paper/2206.14754/

## Key facts

- Cited by 82 papers in the map (data through 14 Oct 2026).
- Builds on 68 of its 100 references that are in the map.
- First posted on arXiv 29 Jun 2022.

## Builds on (most cited first, 30 of 68)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization](https://mlsanity.com/paper/2006.16241/) (2020)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization](https://mlsanity.com/paper/1911.08731/) (2019)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [Counterfactual Fairness](https://mlsanity.com/paper/1703.06856/) (2017)
- [Certifying and removing disparate impact](https://mlsanity.com/paper/1412.3756/) (2014)
- [Network Dissection: Quantifying Interpretability of Deep Visual Representations](https://mlsanity.com/paper/1704.05796/) (2017)
- [Data Shapley: Equitable Valuation of Data for Machine Learning](https://mlsanity.com/paper/1904.02868/) (2019)
- [Interpretable Explanations of Black Boxes by Meaningful Perturbation](https://mlsanity.com/paper/1704.03296/) (2017)
- [Just Train Twice: Improving Group Robustness without Training Group Information](https://mlsanity.com/paper/2107.09044/) (2021)

## Built on by (most cited first, 30 of 82)

- [Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks](https://mlsanity.com/paper/2207.13243/) (2022)
- [Diversify Your Vision Datasets with Automatic Diffusion-Based Augmentation](https://mlsanity.com/paper/2305.16289/) (2023)
- [A Whac-A-Mole Dilemma: Shortcuts Come in Multiples Where Mitigating One Amplifies Others](https://mlsanity.com/paper/2212.04825/) (2022)
- [Dream the Impossible: Outlier Imagination with Diffusion Models](https://mlsanity.com/paper/2309.13415/) (2023)
- [Discover and Cure: Concept-aware Mitigation of Spurious Correlation](https://mlsanity.com/paper/2305.00650/) (2023)
- [Text-To-Concept (and Back) via Cross-Model Alignment](https://mlsanity.com/paper/2305.06386/) (2023)
- [Interpreting CLIP with Hierarchical Sparse Autoencoders](https://mlsanity.com/paper/2502.20578/) (2025)
- [Discovering and Mitigating Visual Biases through Keyword Explanation](https://mlsanity.com/paper/2301.11104/) (2023)
- [Diagnosing and Rectifying Vision Models using Language](https://mlsanity.com/paper/2302.04269/) (2023)
- [Describing Differences in Image Sets with Natural Language](https://mlsanity.com/paper/2312.02974/) (2023)
- [Mitigating Spurious Correlations in Multi-modal Models during Fine-tuning](https://mlsanity.com/paper/2304.03916/) (2023)
- [LANCE: Stress-testing Visual Models by Generating Language-guided Counterfactual Images](https://mlsanity.com/paper/2305.19164/) (2023)
- [Discovering Bugs in Vision Models using Off-the-shelf Image Generation and Captioning](https://mlsanity.com/paper/2208.08831/) (2022)
- [Adaptive Testing of Computer Vision Models](https://mlsanity.com/paper/2212.02774/) (2022)
- [Mass-Producing Failures of Multimodal Systems with Language Models](https://mlsanity.com/paper/2306.12105/) (2023)
- [Dataset Interfaces: Diagnosing Model Failures Using Controllable Counterfactual Generation](https://mlsanity.com/paper/2302.07865/) (2023)
- [PUG: Photorealistic and Semantically Controllable Synthetic Data for Representation Learning](https://mlsanity.com/paper/2308.03977/) (2023)
- [ModelDiff: A Framework for Comparing Learning Algorithms](https://mlsanity.com/paper/2211.12491/) (2022)
- [Multilinear Mixture of Experts: Scalable Expert Specialization through Factorization](https://mlsanity.com/paper/2402.12550/) (2024)
- [FACTS: First Amplify Correlations and Then Slice to Discover Bias](https://mlsanity.com/paper/2309.17430/) (2023)
- [Red Teaming Deep Neural Networks with Feature Synthesis Tools](https://mlsanity.com/paper/2302.10894/) (2023)
- [Effective Human-AI Teams via Learned Natural Language Rules and Onboarding](https://mlsanity.com/paper/2311.01007/) (2023)
- ["Why did the Model Fail?": Attributing Model Performance Changes to Distribution Shifts](https://mlsanity.com/paper/2210.10769/) (2022)
- [Identification of Systematic Errors of Image Classifiers on Rare Subgroups](https://mlsanity.com/paper/2303.05072/) (2023)
- [Slicing Through Bias: Explaining Performance Gaps in Medical Image Analysis using Slice Discovery Methods](https://mlsanity.com/paper/2406.12142/) (2024)
- [LOVM: Language-Only Vision Model Selection](https://mlsanity.com/paper/2306.08893/) (2023)
- [Discover and Mitigate Multiple Biased Subgroups in Image Classifiers](https://mlsanity.com/paper/2403.12777/) (2024)
- [Simplicity Bias of Two-Layer Networks beyond Linearly Separable Data](https://mlsanity.com/paper/2405.17299/) (2024)
- [Failures Are Fated, But Can Be Faded: Characterizing and Mitigating Unwanted Behaviors in Large-Scale Vision and Language Models](https://mlsanity.com/paper/2406.07145/) (2024)
- [HiBug2: Efficient and Interpretable Error Slice Discovery for Comprehensive Model Debugging](https://mlsanity.com/paper/2501.16751/) (2025)

## Abstract (arXiv)

Existing methods for isolating hard subpopulations and spurious correlations in datasets often require human intervention. This can make these methods labor-intensive and dataset-specific. To address these shortcomings, we present a scalable method for automatically distilling a model's failure modes. Specifically, we harness linear classifiers to identify consistent error patterns, and, in turn, induce a natural representation of these failure modes as directions within the feature space. We demonstrate that this framework allows us to discover and automatically caption challenging subpopulations within the training dataset. Moreover, by combining our framework with off-the-shelf diffusion models, we can generate images that are especially challenging for the analyzed model, and thus can be used to perform synthetic data augmentation that helps remedy the model's failure modes. Code available at https://github.com/MadryLab/failure-directions