# Influence Functions for Scalable Data Attribution in Diffusion Models

arXiv: https://arxiv.org/abs/2410.13850 · HTML: https://mlsanity.com/paper/2410.13850/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 36 of its 51 references that are in the map.
- First posted on arXiv 17 Oct 2024.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 30 of 36)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [Variational Diffusion Models](https://mlsanity.com/paper/2107.00630/) (2021)
- [Optimizing Neural Networks with Kronecker-factored Approximate Curvature](https://mlsanity.com/paper/1503.05671/) (2015)
- [Maximum Likelihood Training of Score-Based Diffusion Models](https://mlsanity.com/paper/2101.09258/) (2021)
- [New insights and perspectives on the natural gradient method](https://mlsanity.com/paper/1412.1193/) (2014)
- [Studying Large Language Model Generalization with Influence Functions](https://mlsanity.com/paper/2308.03296/) (2023)
- [TRAK: Attributing Model Behavior at Scale](https://mlsanity.com/paper/2303.14186/) (2023)
- [Limitations of the Empirical Fisher Approximation for Natural Gradient Descent](https://mlsanity.com/paper/1905.12558/) (2019)
- [Generalization in diffusion models arises from geometry-adaptive harmonic representations](https://mlsanity.com/paper/2310.02557/) (2023)
- [On the Accuracy of Influence Functions for Measuring Group Effects](https://mlsanity.com/paper/1905.13289/) (2019)
- [Fast Approximate Natural Gradient Descent in a Kronecker-factored Eigenbasis](https://mlsanity.com/paper/1806.03884/) (2018)
- [If Influence Functions are the Answer, Then What is the Question?](https://mlsanity.com/paper/2209.05364/) (2022)
- [Practical Gauss-Newton Optimisation for Deep Learning](https://mlsanity.com/paper/1706.03662/) (2017)
- [A Kronecker-factored approximate Fisher matrix for convolution layers](https://mlsanity.com/paper/1602.01407/) (2016)
- [DataInf: Efficiently Estimating Data Influence in LoRA-tuned LLMs and Diffusion Models](https://mlsanity.com/paper/2310.00902/) (2023)
- [Which Algorithmic Choices Matter at Which Batch Sizes? Insights From a Noisy Quadratic Model](https://mlsanity.com/paper/1907.04164/) (2019)
- [On Second-Order Group Influence Functions for Black-Box Predictions](https://mlsanity.com/paper/1911.00418/) (2019)
- [The ArtBench Dataset: Benchmarking Generative Models with Artworks](https://mlsanity.com/paper/2206.11404/) (2022)
- [Kronecker-Factored Approximate Curvature for Modern Neural Network Architectures](https://mlsanity.com/paper/2311.00636/) (2023)
- [Intriguing Properties of Data Attribution on Diffusion Models](https://mlsanity.com/paper/2311.00500/) (2023)

## Built on by (most cited first, 30 of 33)

- [A Survey on Mechanistic Interpretability for Multi-Modal Foundation Models](https://mlsanity.com/paper/2502.17516/) (2025)
- [CUPID: Curating Data your Robot Loves with Influence Functions](https://mlsanity.com/paper/2506.19121/) (2025)
- [Diffusion Attribution Score: Evaluating Training Data Influence in Diffusion Models](https://mlsanity.com/paper/2410.18639/) (2024)
- [TokenShapley: Token Level Context Attribution with Shapley Value](https://mlsanity.com/paper/2507.05261/) (2025)
- [Better Training Data Attribution via Better Inverse Hessian-Vector Products](https://mlsanity.com/paper/2507.14740/) (2025)
- [Position: Curvature Matrices Should Be Democratized via Linear Operators](https://mlsanity.com/paper/2501.19183/) (2025)
- [Fast Data Attribution for Text-to-Image Models](https://mlsanity.com/paper/2511.10721/) (2025)
- [Motion Attribution for Video Generation](https://mlsanity.com/paper/2601.08828/) (2026)
- [Nonparametric Data Attribution for Diffusion Models](https://mlsanity.com/paper/2510.14269/) (2025)
- [DMin: Scalable Training Data Influence Estimation for Diffusion Models](https://mlsanity.com/paper/2412.08637/) (2024)
- [Influence Functions for Edge Edits in Non-Convex Graph Neural Networks](https://mlsanity.com/paper/2506.04694/) (2025)
- [Learning to Weight Parameters for Training Data Attribution](https://mlsanity.com/paper/2506.05647/) (2025)
- [Distributional Training Data Attribution: What do Influence Functions Sample?](https://mlsanity.com/paper/2506.12965/) (2025)
- [Concept-TRAK: Understanding how diffusion models learn concepts through concept-level attribution](https://mlsanity.com/paper/2507.06547/) (2025)
- [Better Hessians Matter: Studying the Impact of Curvature Approximations in Influence Functions](https://mlsanity.com/paper/2509.23437/) (2025)
- [The Amazing Stability of Flow Matching](https://mlsanity.com/paper/2604.16079/) (2026)
- [Interaction-Aware Influence Functions for Group Attribution](https://mlsanity.com/paper/2605.15675/) (2026)
- [Generalization vs. Memorization in Autoregressive Deep Learning: Or, Examining Temporal Decay of Gradient Coherence](https://mlsanity.com/paper/2509.00024/) (2025)
- [Sensitivity Analysis for Diffusion Models](https://mlsanity.com/paper/2509.23092/) (2025)
- [On the Accuracy of Newton Step and Influence Function Data Attributions](https://mlsanity.com/paper/2512.12572/) (2025)
- [Unsupervised Synthetic Image Attribution: Alignment and Disentanglement](https://mlsanity.com/paper/2601.22663/) (2026)
- [A Unified Theory of Random Projection for Influence Functions](https://mlsanity.com/paper/2602.10449/) (2026)
- [When Sample Selection Bias Precipitates Model Collapse](https://mlsanity.com/paper/2606.13732/) (2026)
- [ATHENA: Accelerated Multi-Task Heterogeneous Influence Functions for Robot Data Curation](https://mlsanity.com/paper/2606.16208/) (2026)
- [What's a Credit Worth? A Market Framework for Attribution-Aware Compensation in Generative Music](https://mlsanity.com/paper/2607.00641/) (2026)
- [Two-Sample Testing via Path-based Inference](https://mlsanity.com/paper/2610.05684/) (2026)
- [On the Fragility of Data Attribution When Learning Is Distributed](https://mlsanity.com/paper/2605.15520/) (2026)
- [DCS: A Unified Conditional Sensitivity Framework for Cross-Modal Copyright Infringement Detection](https://mlsanity.com/paper/2607.22035/) (2026)
- [Tracing Generated Samples to Training-Data Clusters in Flow-Matching Models](https://mlsanity.com/paper/2608.30081/) (2026)
- [Background Gradients Shape Memorization in Flow Matching](https://mlsanity.com/paper/2609.33210/) (2026)

## Abstract (arXiv)

Diffusion models have led to significant advancements in generative modelling. Yet their widespread adoption poses challenges regarding data attribution and interpretability. In this paper, we aim to help address such challenges in diffusion models by developing an influence functions framework. Influence function-based data attribution methods approximate how a model's output would have changed if some training data were removed. In supervised learning, this is usually used for predicting how the loss on a particular example would change. For diffusion models, we focus on predicting the change in the probability of generating a particular example via several proxy measurements. We show how to formulate influence functions for such quantities and how previously proposed methods can be interpreted as particular design choices in our framework. To ensure scalability of the Hessian computations in influence functions, we systematically develop K-FAC approximations based on generalised Gauss-Newton matrices specifically tailored to diffusion models. We recast previously proposed methods as specific design choices in our framework and show that our recommended method outperforms previous data attribution approaches on common evaluations, such as the Linear Data-modelling Score (LDS) or retraining without top influences, without the need for method-specific hyperparameter tuning.