# Automated Data Slicing for Model Validation:A Big data - AI Integration Approach

arXiv: https://arxiv.org/abs/1807.06068 · HTML: https://mlsanity.com/paper/1807.06068/

## Key facts

- Cited by 70 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jul 2018.

## Built on by (most cited first, 30 of 70)

- [Data-centric Artificial Intelligence: A Survey](https://mlsanity.com/paper/2303.10158/) (2023)
- [Domino: Discovering Systematic Errors with Cross-Modal Embeddings](https://mlsanity.com/paper/2203.14960/) (2022)
- [Salient ImageNet: How to discover spurious features in Deep Learning?](https://mlsanity.com/paper/2110.04301/) (2021)
- [The Spotlight: A General Method for Discovering Systematic Errors in Deep Learning Models](https://mlsanity.com/paper/2107.00758/) (2021)
- [Diagnosing and Rectifying Vision Models using Language](https://mlsanity.com/paper/2302.04269/) (2023)
- [Mandoline: Model Evaluation under Distribution Shift](https://mlsanity.com/paper/2107.00643/) (2021)
- [Mitigating Spurious Correlations in Multi-modal Models during Fine-tuning](https://mlsanity.com/paper/2304.03916/) (2023)
- [Data-centric AI: Perspectives and Challenges](https://mlsanity.com/paper/2301.04819/) (2023)
- [Representation Bias in Data: A Survey on Identification and Resolution Techniques](https://mlsanity.com/paper/2203.11852/) (2022)
- [Improving Human-AI Collaboration With Descriptions of AI Behavior](https://mlsanity.com/paper/2301.06937/) (2023)
- [On the Challenges and Opportunities in Generative AI](https://mlsanity.com/paper/2403.00025/) (2024)
- [Interpretable Data-Based Explanations for Fairness Debugging](https://mlsanity.com/paper/2112.09745/) (2021)
- [Discovering and Validating AI Errors With Crowdsourced Failure Reports](https://mlsanity.com/paper/2109.11690/) (2021)
- [Designerly Understanding: Information Needs for Model Transparency to Support Design Ideation for AI-Powered User Experience](https://mlsanity.com/paper/2302.10395/) (2023)
- [Core Risk Minimization using Salient ImageNet](https://mlsanity.com/paper/2203.15566/) (2022)
- [DC-Check: A Data-Centric AI checklist to guide the development of reliable machine learning systems](https://mlsanity.com/paper/2211.05764/) (2022)
- [AGRO: Adversarial Discovery of Error-prone groups for Robust Optimization](https://mlsanity.com/paper/2212.00921/) (2022)
- [fAIlureNotes: Supporting Designers in Understanding the Limits of AI Models for Computer Vision Tasks](https://mlsanity.com/paper/2302.11703/) (2023)
- [Where Does My Model Underperform? A Human Evaluation of Slice Discovery Algorithms](https://mlsanity.com/paper/2306.08167/) (2023)
- [Unearthing Skill-Level Insights for Understanding Trade-Offs of Foundation Models](https://mlsanity.com/paper/2410.13826/) (2024)
- [Multi-group Learning for Hierarchical Groups](https://mlsanity.com/paper/2402.00258/) (2024)
- [Context-Aware Testing: A New Paradigm for Model Testing with Large Language Models](https://mlsanity.com/paper/2410.24005/) (2024)
- [Identifying Biased Subgroups in Ranking and Classification](https://mlsanity.com/paper/2108.07450/) (2021)
- [Visual Auditor: Interactive Visualization for Detection and Summarization of Model Biases](https://mlsanity.com/paper/2206.12540/) (2022)
- [Is this model reliable for everyone? Testing for strong calibration](https://mlsanity.com/paper/2307.15247/) (2023)
- [AttributionScanner: A Visual Analytics System for Model Validation with Metadata-Free Slice Finding](https://mlsanity.com/paper/2401.06462/) (2024)
- [Tempo: Helping Data Scientists and Domain Experts Collaboratively Specify Predictive Modeling Tasks](https://mlsanity.com/paper/2502.10526/) (2025)
- [Towards a More Rigorous Science of Blindspot Discovery in Image Classification Models](https://mlsanity.com/paper/2207.04104/) (2022)
- [Data-Centric Digital Agriculture: A Perspective](https://mlsanity.com/paper/2312.03437/) (2023)
- [Learning Exceptional Subgroups by End-to-End Maximizing KL-divergence](https://mlsanity.com/paper/2402.12930/) (2024)

## Abstract (arXiv)

As machine learning systems become democratized, it becomes increasingly important to help users easily debug their models. However, current data tools are still primitive when it comes to helping users trace model performance problems all the way to the data. We focus on the particular problem of slicing data to identify subsets of the validation data where the model performs poorly. This is an important problem in model validation because the overall model performance can fail to reflect that of the smaller subsets, and slicing allows users to analyze the model performance on a more granular-level. Unlike general techniques (e.g., clustering) that can find arbitrary slices, our goal is to find interpretable slices (which are easier to take action compared to arbitrary subsets) that are problematic and large. We propose Slice Finder, which is an interactive framework for identifying such slices using statistical techniques. Applications include diagnosing model fairness and fraud detection, where identifying slices that are interpretable to humans is crucial. This research is part of a larger trend of Big data and Artificial Intelligence (AI) integration and opens many opportunities for new research.