# Relabeling Minimal Training Subset to Flip a Prediction

arXiv: https://arxiv.org/abs/2305.12809 · HTML: https://mlsanity.com/paper/2305.12809/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 50 references that are in the map.
- First posted on arXiv 22 May 2023.

## Builds on (most cited first, 23 of 23)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale](https://mlsanity.com/paper/1811.00982/) (2018)
- [Learning the Difference that Makes a Difference with Counterfactually-Augmented Data](https://mlsanity.com/paper/1909.12434/) (2019)
- [Persistent Anti-Muslim Bias in Large Language Models](https://mlsanity.com/paper/2101.05783/) (2021)
- [Machine Unlearning of Features and Labels](https://mlsanity.com/paper/2108.11577/) (2021)
- [Hate Speech Dataset from a White Supremacy Forum](https://mlsanity.com/paper/1809.04444/) (2018)
- [Social Biases in NLP Models as Barriers for Persons with Disabilities](https://mlsanity.com/paper/2005.00813/) (2020)
- [On the Accuracy of Influence Functions for Measuring Group Effects](https://mlsanity.com/paper/1905.13289/) (2019)
- [Debugging Tests for Model Explanations](https://mlsanity.com/paper/2011.05429/) (2020)
- [Explaining Black Box Predictions and Unveiling Data Artifacts through Influence Functions](https://mlsanity.com/paper/2005.06676/) (2020)
- [Truthful AI: Developing and governing AI that does not lie](https://mlsanity.com/paper/2110.06674/) (2021)
- [Input Similarity from the Neural Network Perspective](https://mlsanity.com/paper/2102.05262/) (2021)
- [The Devil of Face Recognition is in the Noise](https://mlsanity.com/paper/1807.11649/) (2018)
- [Interactive Label Cleaning with Example-based Explanations](https://mlsanity.com/paper/2106.03922/) (2021)
- [Summarizing, Simplifying, and Synthesizing Medical Evidence Using GPT-3 (with Varying Success)](https://mlsanity.com/paper/2305.06299/) (2023)
- [Combining Feature and Instance Attribution to Detect Artifacts](https://mlsanity.com/paper/2107.00323/) (2021)
- [An Empirical Comparison of Instance Attribution Methods for NLP](https://mlsanity.com/paper/2104.04128/) (2021)
- [An Automatic Finite-Sample Robustness Metric: When Can Dropping a Little Data Make a Big Difference?](https://mlsanity.com/paper/2011.14999/) (2020)
- [Disentangling Influence: Using Disentangled Representations to Audit Model Predictions](https://mlsanity.com/paper/1906.08652/) (2019)
- [How Many and Which Training Points Would Need to be Removed to Flip this Prediction?](https://mlsanity.com/paper/2302.02169/) (2023)

## Built on by (most cited first, 3 of 3)

- [How to Achieve Higher Accuracy with Less Training Points?](https://mlsanity.com/paper/2504.13586/) (2025)
- [Detecting Mislabeled and Corrupted Data via Pointwise Mutual Information](https://mlsanity.com/paper/2508.07713/) (2025)
- [On Robustness of Linear Classifiers to Targeted Data Poisoning](https://mlsanity.com/paper/2511.12722/) (2025)

## Abstract (arXiv)

When facing an unsatisfactory prediction from a machine learning model, users can be interested in investigating the underlying reasons and exploring the potential for reversing the outcome. We ask: To flip the prediction on a test point xt, how to identify the smallest training subset St that we need to relabel? We propose an efficient algorithm to identify and relabel such a subset via an extended influence function for binary classification models with convex loss. We find that relabeling fewer than 2% of the training points can always flip a prediction. This mechanism can serve multiple purposes: (1) providing an approach to challenge a model prediction by altering training points; (2) evaluating model robustness with the cardinality of the subset (i.e., |St|); we show that |St| is highly related to the noise ratio in the training set and |St| is correlated with but complementary to predicted probabilities; and (3) revealing training points lead to group attribution bias. To the best of our knowledge, we are the first to investigate identifying and relabeling the minimal training subset required to flip a given prediction.