mlsanityData to 14 Oct 2026

MAGIC: Near-Optimal Data Attribution for Deep Learning

arXiv 2504.16430 ↗First posted 23 Apr 2025cs.LG cs.CL cs.CVcited by 22

Not on the map. Why?

Abstract (arXiv)

The goal of predictive data attribution is to estimate how adding or removing a given set of training datapoints will affect model predictions. In convex settings, this goal is straightforward (i.e., via the infinitesimal jackknife). In large-scale (non-convex) settings, however, existing methods are far less successful -- current methods' estimates often only weakly correlate with ground truth. In this work, we present a new data attribution method (MAGIC) that combines classical methods and recent advances in metadifferentiation to (nearly) optimally estimate the effect of adding or removing training data on model predictions.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Builds on 34 of its 46 references that are in the map. First posted on arXiv 23 Apr 2025.

Builds on

34 of 46 references.

References · 34

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2020Scaling Laws for Neural Language Modelscited by 6,586
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  6. 2017Understanding Black-box Predictions via Influence Functionscited by 1,741
  7. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,319
  8. 2024Gemma: Open Models Based on Gemini Research and Technologycited by 916
  9. 2015Optimizing Neural Networks with Kronecker-factored Approximate Curvaturecited by 796
  10. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 739
  11. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 721
  12. 2024LESS: Selecting Influential Data for Targeted Instruction Tuningcited by 532
22 more
  1. 2023Scaling Data-Constrained Language Modelscited by 403
  2. 2015Gradient-based Hyperparameter Optimization through Reversible Learningcited by 360
  3. 2023Studying Large Language Model Generalization with Influence Functionscited by 326
  4. 2023TRAK: Attributing Model Behavior at Scalecited by 283
  5. 2020Influence Functions in Deep Learning Are Fragilecited by 252
  6. 2017Forward and Reverse Gradient-Based Hyperparameter Optimizationcited by 234
  7. 2022Datamodels: Predicting Predictions from Training Datacited by 189
  8. 2019On the Accuracy of Influence Functions for Measuring Group Effectscited by 147
  9. 2018Fast Approximate Natural Gradient Descent in a Kronecker-factored Eigenbasiscited by 146
  10. 2022If Influence Functions are the Answer, Then What is the Question?cited by 144
  11. 2022Training Data Influence Analysis and Estimation: A Surveycited by 135
  12. 2021Randomness In Neural Network Training: Characterizing The Impact of Toolingcited by 59
  13. 2023Simfluence: Modeling the Influence of Individual Training Examples by Simulating Training Runscited by 53
  14. 2024Training Data Attribution via Approximate Unrolled Differentiationcited by 34
  15. 2022ModelDiff: A Framework for Comparing Learning Algorithmscited by 30
  16. 2022Understanding Influence Functions and Datamodels via Harmonic Analysiscited by 23
  17. 2025Optimizing ML Training with Metagradient Descentcited by 23
  18. 2023On the Variance of Neural Network Training with respect to Test Sets and Distributionscited by 22
  19. 2024Attribute-to-Delete: Machine Unlearning via Datamodel Matchingcited by 18
  20. 2018A scalable estimate of the extra-sample prediction error via approximate leave-one-outcited by 11
  21. 2024Generalized Group Data Attributioncited by 6
  22. 2024RandALO: Out-of-sample risk estimation in no time flatcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 17.

22 papers.

Sort:

By area:

  1. 2025CUPID: Curating Data your Robot Loves with Influence Functionscited by 23
  2. 2026Subliminal Effects in Your Data: A General Mechanism via Log-Linearitycited by 19
  3. 2025Better Training Data Attribution via Better Inverse Hessian-Vector Productscited by 10
  4. 2025Train Once, Answer All: Many Pretraining Experiments for the Cost of Onecited by 6
  5. 2026Infusion: Shaping Model Behavior by Editing Training Data via Influence Functionscited by 5
  6. 2026Efficient Estimation of Kernel Surrogate Models for Task Attributioncited by 4
  7. 2026How Faithful Is Trajectory-Based Data Attribution? Error Sources, Remedies, and Practical Guidelinescited by 4
  8. 2025Distributional Training Data Attribution: What do Influence Functions Sample?cited by 3
  9. 2025Better Hessians Matter: Studying the Impact of Curvature Approximations in Influence Functionscited by 3
  10. 2026In-Run Data Shapley for Adam Optimizercited by 3
  11. 2026Position: Don't Just "Fix it in Post": A Science of AI Must Study Training Dynamicscited by 3
  12. 2026Bergson: An Open Source Library for Data Attributioncited by 3
10 more
  1. 2026How to sketch a learning algorithmcited by 2
  2. 2026Prototype Language Modelscited by 2
  3. 2026Challenges in Enabling Private Data Valuationcited by 1
  4. 2026Curation Leaks: Membership Inference Attacks against Data Curation for Machine Learningcited by 0
  5. 2026Small edits, large models: How Wikipedia advocacy shapes LLM valuescited by 0
  6. 2026Assert, don't describe: Linguistic features that shift LLM reasoning about animal welfarecited by 0
  7. 2026Data Attribution via Sketched Metadifferentiationcited by 0
  8. 2026Scalable Attribution and Control of Model Behavior During Trainingcited by 0
  9. 2026dattri-LLM: A Unified and Efficient Library for Training Data Attribution at LLM Scalecited by 0
  10. 2026Framework for an AI-Ready ARPES Community Databasecited by 0