# Where Does My Model Underperform? A Human Evaluation of Slice Discovery Algorithms

arXiv: https://arxiv.org/abs/2306.08167 · HTML: https://mlsanity.com/paper/2306.08167/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 51 references that are in the map.
- First posted on arXiv 13 Jun 2023.

## Builds on (most cited first, 26 of 26)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution](https://mlsanity.com/paper/2202.10054/) (2022)
- [When and why vision-language models behave like bags-of-words, and what to do about it?](https://mlsanity.com/paper/2210.01936/) (2022)
- [Just Train Twice: Improving Group Robustness without Training Group Information](https://mlsanity.com/paper/2107.09044/) (2021)
- [Improving fairness in machine learning systems: What do industry practitioners need?](https://mlsanity.com/paper/1812.05239/) (2018)
- [No Subclass Left Behind: Fine-Grained Robustness in Coarse-Grained Classification Problems](https://mlsanity.com/paper/2011.12945/) (2020)
- [Simple data balancing achieves competitive worst-group-accuracy](https://mlsanity.com/paper/2110.14503/) (2021)
- [Domino: Discovering Systematic Errors with Cross-Modal Embeddings](https://mlsanity.com/paper/2203.14960/) (2022)
- [A Whac-A-Mole Dilemma: Shortcuts Come in Multiples Where Mitigating One Amplifies Others](https://mlsanity.com/paper/2212.04825/) (2022)
- [Automated Data Slicing for Model Validation:A Big data - AI Integration Approach](https://mlsanity.com/paper/1807.06068/) (2018)
- [The Spotlight: A General Method for Discovering Systematic Errors in Deep Learning Models](https://mlsanity.com/paper/2107.00758/) (2021)
- [When does dough become a bagel? Analyzing the remaining mistakes on ImageNet](https://mlsanity.com/paper/2205.04596/) (2022)
- [Post hoc Explanations may be Ineffective for Detecting Unknown Spurious Correlation](https://mlsanity.com/paper/2212.04629/) (2022)
- [Understanding Failures of Deep Networks via Robust Feature Extraction](https://mlsanity.com/paper/2012.01750/) (2020)
- [Discovering Bugs in Vision Models using Off-the-shelf Image Generation and Captioning](https://mlsanity.com/paper/2208.08831/) (2022)
- [Program Synthesis using Natural Language](https://mlsanity.com/paper/1509.00413/) (2015)
- [Adaptive Testing of Computer Vision Models](https://mlsanity.com/paper/2212.02774/) (2022)
- [Dataset Interfaces: Diagnosing Model Failures Using Controllable Counterfactual Generation](https://mlsanity.com/paper/2302.07865/) (2023)
- [Using Language to Extend to Unseen Domains](https://mlsanity.com/paper/2210.09520/) (2022)
- [Zeno: An Interactive Framework for Behavioral Evaluation of Machine Learning](https://mlsanity.com/paper/2302.04732/) (2023)
- [Discovering and Validating AI Errors With Crowdsourced Failure Reports](https://mlsanity.com/paper/2109.11690/) (2021)
- [SEAL : Interactive Tool for Systematic Error Analysis and Labeling](https://mlsanity.com/paper/2210.05839/) (2022)
- [DendroMap: Visual Exploration of Large-Scale Image Datasets for Machine Learning with Treemaps](https://mlsanity.com/paper/2205.06935/) (2022)
- [fAIlureNotes: Supporting Designers in Understanding the Limits of AI Models for Computer Vision Tasks](https://mlsanity.com/paper/2302.11703/) (2023)

## Built on by (most cited first, 9 of 9)

- [Multicalibration for Confidence Scoring in LLMs](https://mlsanity.com/paper/2404.04689/) (2024)
- [HiBug2: Efficient and Interpretable Error Slice Discovery for Comprehensive Model Debugging](https://mlsanity.com/paper/2501.16751/) (2025)
- [PRIME: Prioritizing Interpretability in Failure Mode Extraction](https://mlsanity.com/paper/2310.00164/) (2023)
- [Policy Maps: Tools for Guiding the Unbounded Space of LLM Behaviors](https://mlsanity.com/paper/2409.18203/) (2024)
- [VISLIX: An XAI Framework for Validating Vision Models with Slice Discovery and Analysis](https://mlsanity.com/paper/2505.03132/) (2025)
- [TRoVe: Discovering Error-Inducing Static Feature Biases in Temporal Vision-Language Models](https://mlsanity.com/paper/2512.01048/) (2025)
- [What Is Wrong with My Model? Identifying Systematic Problems with Semantic Data Slicing](https://mlsanity.com/paper/2409.09261/) (2024)
- [Data Heterogeneity Modeling for Trustworthy Machine Learning](https://mlsanity.com/paper/2506.00969/) (2025)
- [Intersectional Disentangling of Temporal and Acquisition Bias in Fetal Ultrasound](https://mlsanity.com/paper/2605.02942/) (2026)

## Abstract (arXiv)

Machine learning (ML) models that achieve high average accuracy can still underperform on semantically coherent subsets ("slices") of data. This behavior can have significant societal consequences for the safety or bias of the model in deployment, but identifying these underperforming slices can be difficult in practice, especially in domains where practitioners lack access to group annotations to define coherent subsets of their data. Motivated by these challenges, ML researchers have developed new slice discovery algorithms that aim to group together coherent and high-error subsets of data. However, there has been little evaluation focused on whether these tools help humans form correct hypotheses about where (for which groups) their model underperforms. We conduct a controlled user study (N = 15) where we show 40 slices output by two state-of-the-art slice discovery algorithms to users, and ask them to form hypotheses about an object detection model. Our results provide positive evidence that these tools provide some benefit over a naive baseline, and also shed light on challenges faced by users during the hypothesis formation step. We conclude by discussing design opportunities for ML and HCI researchers. Our findings point to the importance of centering users when creating and evaluating new tools for slice discovery.