# DeMix: Debugging Training Data with Mixed Data Error Types by Investigating Influence Vectors

arXiv: https://arxiv.org/abs/2606.11616 · HTML: https://mlsanity.com/paper/2606.11616/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 44 references that are in the map.
- First posted on arXiv 10 Jun 2026.

## Builds on (most cited first, 21 of 21)

- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [Deep Variational Information Bottleneck](https://mlsanity.com/paper/1612.00410/) (2016)
- [Deep Interest Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1706.06978/) (2017)
- [LESS: Selecting Influential Data for Targeted Instruction Tuning](https://mlsanity.com/paper/2402.04333/) (2024)
- [Just Train Twice: Improving Group Robustness without Training Group Information](https://mlsanity.com/paper/2107.09044/) (2021)
- [Deep Interest Evolution Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1809.03672/) (2018)
- [Interpretable and Generalizable Graph Learning via Stochastic Attention Mechanism](https://mlsanity.com/paper/2201.12987/) (2022)
- [ODIN: Disentangled Reward Mitigates Hacking in RLHF](https://mlsanity.com/paper/2402.07319/) (2024)
- [Training Data Influence Analysis and Estimation: A Survey](https://mlsanity.com/paper/2212.04612/) (2022)
- [Discover and Cure: Concept-aware Mitigation of Spurious Correlation](https://mlsanity.com/paper/2305.00650/) (2023)
- [From Lists to Emojis: How Format Bias Affects Model Alignment](https://mlsanity.com/paper/2409.11704/) (2024)
- [Dataset Regeneration for Sequential Recommendation](https://mlsanity.com/paper/2405.17795/) (2024)
- [DataMan: Data Manager for Pre-training Large Language Models](https://mlsanity.com/paper/2502.19363/) (2025)
- [Back to the Basics: Revisiting Out-of-Distribution Detection Baselines](https://mlsanity.com/paper/2207.03061/) (2022)
- [A Comprehensive Survey on Imbalanced Data Learning](https://mlsanity.com/paper/2502.08960/) (2025)
- [Error Discovery by Clustering Influence Embeddings](https://mlsanity.com/paper/2312.04712/) (2023)
- [Influence Guided Context Selection for Effective Retrieval-Augmented Generation](https://mlsanity.com/paper/2509.21359/) (2025)
- [Shapley Value-driven Data Pruning for Recommender Systems](https://mlsanity.com/paper/2505.22057/) (2025)
- [Improving Group Robustness on Spurious Correlation via Evidential Alignment](https://mlsanity.com/paper/2506.11347/) (2025)

## Built on by (most cited first, 1 of 1)

- [Groups Don't Move as One: A Bregman Clustering View of Linearized Group Influence](https://mlsanity.com/paper/2610.14809/) (2026)

## Abstract (arXiv)

High-quality training data is essential for the success of machine learning models. However, real-world datasets often contain mixed types of errors arising from systematic flaws in data preparation pipelines, including label errors, feature errors, and spurious correlations. Effective debugging of training data requires both detecting erroneous samples and identifying their specific error types to enable targeted repair, yet existing data cleaning and attribution methods fail to adequately address this dual requirement. In this paper, we propose DeMix, a novel framework that simultaneously diagnoses erroneous samples and their error types. Our key insight is that different error types produce distinct patterns on model behavior. DeMix captures such error-specific patterns by influence vectors that characterize how each training sample affects model predictions across all validation samples. We formulate training data debugging as a multi-label classification problem where a classifier is developed to predict error types directly from influence vectors. We further introduce an intervention-based learning strategy that guides the classifier to capture invariant rationales specific to each error type, ensuring the learned classifier generalizes effectively. Empirical evaluations on 11 tasks across tabular data prediction, recommendation systems, and LLM alignment demonstrate that DeMix significantly outperforms state-of-the-art approaches, achieving a 22.61% improvement in data debugging F1-score and a 9.32% gain in task model performance after data repair. Code is available at: https://github.com/SJTU-DMTai/DeMix.