# Z0-Inf: Zeroth Order Approximation for Data Influence

arXiv: https://arxiv.org/abs/2510.11832 · HTML: https://mlsanity.com/paper/2510.11832/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 28 references that are in the map.
- First posted on arXiv 13 Oct 2025.

## Builds on (most cited first, 20 of 20)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [Data Shapley: Equitable Valuation of Data for Machine Learning](https://mlsanity.com/paper/1904.02868/) (2019)
- [Estimating Training Data Influence by Tracing Gradient Descent](https://mlsanity.com/paper/2002.08484/) (2020)
- [LESS: Selecting Influential Data for Targeted Instruction Tuning](https://mlsanity.com/paper/2402.04333/) (2024)
- [What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation](https://mlsanity.com/paper/2008.03703/) (2020)
- [Deep Layer Aggregation](https://mlsanity.com/paper/1707.06484/) (2017)
- [Studying Large Language Model Generalization with Influence Functions](https://mlsanity.com/paper/2308.03296/) (2023)
- [TRAK: Attributing Model Behavior at Scale](https://mlsanity.com/paper/2303.14186/) (2023)
- [Influence Functions in Deep Learning Are Fragile](https://mlsanity.com/paper/2006.14651/) (2020)
- [Scaling Up Influence Functions](https://mlsanity.com/paper/2112.03052/) (2021)
- [DataInf: Efficiently Estimating Data Influence in LoRA-tuned LLMs and Diffusion Models](https://mlsanity.com/paper/2310.00902/) (2023)
- [Data Shapley in One Training Run](https://mlsanity.com/paper/2406.11011/) (2024)
- [Data Cleansing for Models Trained with SGD](https://mlsanity.com/paper/1906.08473/) (2019)
- [Revisiting Methods for Finding Influential Examples](https://mlsanity.com/paper/2111.04683/) (2021)
- [Training Data Attribution via Approximate Unrolled Differentiation](https://mlsanity.com/paper/2405.12186/) (2024)
- [Theoretical and Practical Perspectives on what Influence Functions Do](https://mlsanity.com/paper/2305.16971/) (2023)
- [Do Influence Functions Work on Large Language Models?](https://mlsanity.com/paper/2409.19998/) (2024)
- [textttdattri: A Library for Efficient Data Attribution](https://mlsanity.com/paper/2410.04555/) (2024)

## Built on by (most cited first, 3 of 3)

- [Sketching the Readout of Large Language Models for Scalable Data Attribution and Valuation](https://mlsanity.com/paper/2604.16197/) (2026)
- [From Detrimental to Beneficial: Dynamic Influence-based Valuation and Editing](https://mlsanity.com/paper/2608.22522/) (2026)
- [Not All Rollouts Are Worth Learning: On Trajectory Valuation for Post-Training Reinforcement Learning](https://mlsanity.com/paper/2609.35072/) (2026)

## Abstract (arXiv)

A critical aspect of analyzing and improving modern machine learning systems lies in understanding how individual training examples influence a model's predictive behavior. Estimating this influence enables critical applications, including data selection and model debugging; in particular, self-influence, which quantifies the influence of a training point on itself, has found many uses in data quality assessment and outlier detection. Existing methods for measuring data influence, however, are often impractical for large models due to low accuracy or prohibitive computational costs: most approaches either provide poor approximations or rely on gradients and inverse-Hessian computations that remain challenging to scale. In this work, we introduce a highly efficient zeroth-order approximation for estimating the influence of training data that requires only a fraction of the time and memory footprint of prior methods. Notably, our method relies solely on loss values of intermediate checkpoints on the training and test data, along with the checkpoints themselves, making it broadly applicable even when the loss function of interest is non-differentiable. Beyond its computational efficiency, our approach achieves superior accuracy in estimating self-influence and comparable or improved accuracy in estimating train-test influence for fine-tuned large language models, enabling scalable and practical analysis of how training data shapes model behavior.