# Slice-based Learning: A Programming Model for Residual Learning in Critical Data Slices

arXiv: https://arxiv.org/abs/1909.06349 · HTML: https://mlsanity.com/paper/1909.06349/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Sep 2019.

## Built on by (most cited first, 23 of 23)

- [Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Models](https://mlsanity.com/paper/2101.00288/) (2021)
- [Lifting the Curse of Multilinguality by Pre-training Modular Transformers](https://mlsanity.com/paper/2205.06266/) (2022)
- [Robustness Gym: Unifying the NLP Evaluation Landscape](https://mlsanity.com/paper/2101.04840/) (2021)
- [Mandoline: Model Evaluation under Distribution Shift](https://mlsanity.com/paper/2107.00643/) (2021)
- [Discovering and Validating AI Errors With Crowdsourced Failure Reports](https://mlsanity.com/paper/2109.11690/) (2021)
- [mmT5: Modular Multilingual Pre-Training Solves Source Language Hallucinations](https://mlsanity.com/paper/2305.14224/) (2023)
- [ScatterShot: Interactive In-context Example Curation for Text Transformation](https://mlsanity.com/paper/2302.07346/) (2023)
- [Wikibench: Community-Driven Data Curation for AI Evaluation on Wikipedia](https://mlsanity.com/paper/2402.14147/) (2024)
- [Deep Learning Safety Concerns in Automated Driving Perception](https://mlsanity.com/paper/2309.03774/) (2023)
- [VLSlice: Interactive Vision-and-Language Slice Discovery](https://mlsanity.com/paper/2309.06703/) (2023)
- [Managing ML Pipelines: Feature Stores and the Coming Wave of Embedding Ecosystems](https://mlsanity.com/paper/2108.05053/) (2021)
- [The Lifecycle of a Statistical Model: Model Failure Detection, Identification, and Refitting](https://mlsanity.com/paper/2202.04166/) (2022)
- [Handling Long-Tail Queries with Slice-Aware Conversational Systems](https://mlsanity.com/paper/2104.13216/) (2021)
- [A Conceptual Framework for Ethical Evaluation of Machine Learning Systems](https://mlsanity.com/paper/2408.10239/) (2024)
- [Label Augmentation with Reinforced Labeling for Weak Supervision](https://mlsanity.com/paper/2204.06436/) (2022)
- [AutoSlicer: Scalable Automated Data Slicing for ML Model Analysis](https://mlsanity.com/paper/2212.09032/) (2022)
- [Fusing Conditional Submodular GAN and Programmatic Weak Supervision](https://mlsanity.com/paper/2312.10366/) (2023)
- [PyTDC: A multimodal machine learning training, evaluation, and inference platform for biomedical foundation models](https://mlsanity.com/paper/2505.05577/) (2025)
- [Learning Slice-Aware Representations with Mixture of Attentions](https://mlsanity.com/paper/2106.02363/) (2021)
- [Metadata Shaping: Natural Language Annotations for the Tail](https://mlsanity.com/paper/2110.08430/) (2021)
- [Generative Modeling Helps Weak Supervision (and Vice Versa)](https://mlsanity.com/paper/2203.12023/) (2022)
- [A Causal Framework for Aligning Image Quality Metrics and Deep Neural Network Robustness](https://mlsanity.com/paper/2503.02797/) (2025)
- [ESPO: Error-Structured Prompt Optimization via Diagnose, Diversify, and Stabilize](https://mlsanity.com/paper/2609.04197/) (2026)

## Abstract (arXiv)

In real-world machine learning applications, data subsets correspond to especially critical outcomes: vulnerable cyclist detections are safety-critical in an autonomous driving task, and "question" sentences might be important to a dialogue agent's language understanding for product purposes. While machine learning models can achieve high quality performance on coarse-grained metrics like F1-score and overall accuracy, they may underperform on critical subsets---we define these as slices, the key abstraction in our approach. To address slice-level performance, practitioners often train separate "expert" models on slice subsets or use multi-task hard parameter sharing. We propose Slice-based Learning, a new programming model in which the slicing function (SF), a programming interface, specifies critical data subsets for which the model should commit additional capacity. Any model can leverage SFs to learn slice expert representations, which are combined with an attention mechanism to make slice-aware predictions. We show that our approach maintains a parameter-efficient representation while improving over baselines by up to 19.0 F1 on slices and 4.6 F1 overall on datasets spanning language understanding (e.g. SuperGLUE), computer vision, and production-scale industrial systems.