mlsanityData to 14 Oct 2026

Traceprop: Training Data Attribution in a Single Pass

arXiv 2609.32380 ↗First posted 26 Sep 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Data attribution tools used in practice, TRAK, LoGRA/LogIX, and EK-FAC, run after training: they recompute per-sample gradients in a separate pass over the training set, and curvature-aware variants need a further pass to estimate covariance. Traceprop records projected per-sample gradients and K-FAC covariance statistics inside the training backward pass itself. On Pythia-1B LoRA fine-tunes on an A100, this adds 2.6% to training time, against 10.7% for LogIX's best inline configuration, random-init (4.1x lower, one-sided Mann-Whitney p = 9e-5, n = 10 per arm); at Pythia-6.9B the numbers are 11.3% and 27.0% (p = 0.004). LogIX's default configuration, PCA-init, is both slower and lower quality than random-init, so we compare against random-init throughout. On a small transformer where LDS is measurable, Traceprop matches LogIX's best configuration: pooled LDS difference +0.0022, 95% CI [-0.0038, +0.0075]. Two things do not work: at Pythia-160M/SST-2, every method is indistinguishable from a low noise ceiling, and on planted-backdoor and mislabel detection, gradient attribution does not beat gradient norm or representation similarity. The contribution here is systems, not a new estimator: LogIX's attribution quality, in one pass instead of two.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 16 of its 37 references that are in the map. First posted on arXiv 26 Sep 2026.

Builds on

16 of 37 references.

References · 16

  1. 2017Understanding Black-box Predictions via Influence Functionscited by 1,741
  2. 2019Machine Unlearningcited by 1,061
  3. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 864
  4. 2020Estimating Training Data Influence by Tracing Gradient Descentcited by 580
  5. 2024LESS: Selecting Influential Data for Targeted Instruction Tuningcited by 532
  6. 2019Does Learning Require Memorization? A Short Tale about a Long Tailcited by 460
  7. 2021Remember What You Want to Forget: Algorithms for Machine Unlearningcited by 371
  8. 2023Studying Large Language Model Generalization with Influence Functionscited by 326
  9. 2021Machine Unlearning of Features and Labelscited by 229
  10. 2024DsDm: Model-Aware Dataset Selection with Datamodelscited by 105
  11. 2024What is Your Data Worth to GPT? LLM-Scale Data Valuation with Influence Functionscited by 94
  12. 2024Data Shapley in One Training Runcited by 81
4 more
  1. 2024Scalable Influence and Fact Tracing for Large Language Model Pretrainingcited by 42
  2. 2024textttdattri: A Library for Efficient Data Attributioncited by 18
  3. 2022Management of Machine Learning Lifecycle Artifacts: A Surveycited by 13
  4. 2018Smoke: Fine-grained Lineage at Interactive Speedcited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023TRAK: Attributing Model Behavior at Scale
  2. 2026Data Attribution via Sketched Metadifferentiation
  3. 2025Nonparametric Data Attribution for Diffusion Models
  4. 2026SteerCheck: Attribution Specificity and Alignment Leakage in Activation-Steering Audits
  5. 2026WildTrace: Evaluating Training Data Detection in Realistic Large-Scale LLM Post-Training
  6. 2026Scalable Attribution and Control of Model Behavior During Training
  7. 2026From Behavior to Provenance: Attributing Tabular Foundation Models to Synthetic Pretraining Data
  8. 2024Diffusion Attribution Score: Evaluating Training Data Influence in Diffusion Models