mlsanityData to 7 Oct 2026

The Journey, Not the Destination: How Data Guides Diffusion Models

arXiv 2312.06205 ↗First posted 11 Dec 2023cs.CV cs.LGcited by 42

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion models trained on large datasets can synthesize photo-realistic images of remarkable quality and diversity. However, attributing these images back to the training data-that is, identifying specific training examples which caused an image to be generated-remains a challenge. In this paper, we propose a framework that: (i) provides a formal notion of data attribution in the context of diffusion models, and (ii) allows us to counterfactually validate such attributions. Then, we provide a method for computing these attributions efficiently. Finally, we apply our method to find (and evaluate) such attributions for denoising diffusion probabilistic models trained on CIFAR-10 and latent diffusion models trained on MS COCO. We provide code at https://github.com/MadryLab/journey-TRAK .

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 49 of its 60 references that are in the map. First posted on arXiv 11 Dec 2023.

Builds on

49 of 60 references.

Closest ancestors · 6

Same direction, newest first

  1. 2023Consistency Models2 Mar
  2. 2023Consistent Diffusion Models: Mitigating Sampling Drift by Learning to be Consistent17 Feb
  3. 2022Classifier-Free Diffusion Guidance26 Jul
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  6. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 43

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  6. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  7. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  8. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  9. 2020Extracting Training Data from Large Language Modelscited by 2,144
  10. 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
  11. 2023Extracting Training Data from Diffusion Modelscited by 723
  12. 2019Data Shapley: Equitable Valuation of Data for Machine Learningcited by 617
31 more
  1. 2020Estimating Training Data Influence by Tracing Gradient Descentcited by 565
  2. 2020What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimationcited by 492
  3. 2022Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Modelscited by 418
  4. 2023Synthetic Data from Diffusion Models Improves ImageNet Classificationcited by 326
  5. 2019Towards Efficient Data Valuation Based on the Shapley Valuecited by 279
  6. 2023TRAK: Attributing Model Behavior at Scalecited by 273
  7. 2023Stable Bias: Analyzing Societal Representations in Diffusion Modelscited by 225
  8. 2018Representer Point Selection for Explaining Deep Neural Networkscited by 187
  9. 2022Datamodels: Predicting Predictions from Training Datacited by 186
  10. 2019On the Accuracy of Influence Functions for Measuring Group Effectscited by 146
  11. 2022If Influence Functions are the Answer, Then What is the Question?cited by 142
  12. 2022Training Data Influence Analysis and Estimation: A Surveycited by 134
  13. 2021Scaling Up Influence Functionscited by 130
  14. 2022A Kernel-Based View of Language Model Fine-Tuningcited by 118
  15. 2021Neural Networks as Kernel Learners: The Silent Alignment Effectcited by 106
  16. 2022More Than a Toy: Random Matrix Models Predict How Real-World Neural Representations Generalizecited by 87
  17. 2019On Second-Order Group Influence Functions for Black-Box Predictionscited by 78
  18. 2021On Memorization in Probabilistic Deep Generative Modelscited by 60
  19. 2022Measuring the Effect of Training Data on Deep Learning Predictions via Randomized Experimentscited by 50
  20. 2018Interpreting Black Box Predictions using Fisher Kernelscited by 48
  21. 2022Discovering Bugs in Vision Models using Off-the-shelf Image Generation and Captioningcited by 45
  22. 2023Analyzing Bias in Diffusion-based Face Generation Modelscited by 45
  23. 2023Evaluating Data Attribution for Text-to-Image Modelscited by 45
  24. 2023Dataset Interfaces: Diagnosing Model Failures Using Controllable Counterfactual Generationcited by 35
  25. 2022When do GANs replicate? On the choice of dataset sizecited by 34
  26. 2022ModelDiff: A Framework for Comparing Learning Algorithmscited by 30
  27. 2021Properties of the After Kernelcited by 27
  28. 2020LQF: Linear Quadratic Fine-Tuningcited by 25
  29. 2016"Influence Sketching": Finding Influential Samples In Large-Scale Regressionscited by 17
  30. 2022Data Debugging with Shapley Importance over End-to-End Machine Learning Pipelinescited by 16
  31. 2022Invariant Learning via Diffusion Dreamed Distribution Shiftscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 19; 2025: 10; 2026: 13.

42 papers.

Sort:

By area:

  1. 2024ConceptPrune: Concept Editing in Diffusion Models via Skilled Neuron Pruningcited by 51
  2. 2024Critical windows: non-asymptotic theory for feature emergence in diffusion modelscited by 41
  3. 2024Training Data Attribution via Approximate Unrolled Differentiationcited by 33
  4. 2024Influence Functions for Scalable Data Attribution in Diffusion Modelscited by 31
  5. 2024Data Attribution for Text-to-Image Models by Unlearning Synthesized Imagescited by 22
  6. 2024Diffusion Attribution Score: Evaluating Training Data Influence in Diffusion Modelscited by 21
  7. 2025CUPID: Curating Data your Robot Loves with Influence Functionscited by 21
  8. 2024Decomposing and Editing Predictions by Modeling Model Computationcited by 19
  9. 2024An Economic Solution to Copyright Challenges of Generative AIcited by 18
  10. 2025Selective Underfitting in Diffusion Modelscited by 17
  11. 2024DiffusionPID: Interpreting Diffusion via Partial Information Decompositioncited by 16
  12. 2024Data Attribution for Diffusion Models: Timestep-induced Bias in Influence Estimationcited by 13
30 more
  1. 2024A Survey of Defenses Against AI-Generated Visual Media: Detection,Disruption, and Authenticationcited by 12
  2. 2026Making Reconstruction FID Predictive of Diffusion Generation FIDcited by 12
  3. 2024Replication in Visual Diffusion Models: A Survey and Outlookcited by 11
  4. 2024A Survey on Responsible Generative AI: What to Generate and What Notcited by 8
  5. 2024A LoRA is Worth a Thousand Picturescited by 7
  6. 2025Fast Data Attribution for Text-to-Image Modelscited by 7
  7. 2026Theory of Speciation Transitions in Diffusion Models with General Class Structurecited by 7
  8. 2024CopyrightShield: Enhancing Diffusion Model Security against Copyright Infringement Attackscited by 5
  9. 2024An Efficient Framework for Crediting Data Contributors of Diffusion Modelscited by 4
  10. 2026Motion Attribution for Video Generationcited by 4
  11. 2024DMin: Scalable Training Data Influence Estimation for Diffusion Modelscited by 3
  12. 2025Learning to Weight Parameters for Training Data Attributioncited by 3
  13. 2024Elucidating Flow Matching ODE Dynamics with Respect to Data Geometries and Denoiserscited by 2
  14. 2025Not Every Time and Frequency Need to Be Forgotten in Diffusion Unlearningcited by 2
  15. 2024Enhancing High-Energy Particle Physics Collision Analysis through Graph Data Attribution Techniquescited by 1
  16. 2025Small-to-Large Generalization: Data Influences Models Consistently Across Scalecited by 1
  17. 2025Identifying Prompted Artist Names from Generated Imagescited by 1
  18. 2025Sensitivity Analysis for Diffusion Modelscited by 1
  19. 2026Unsupervised Synthetic Image Attribution: Alignment and Disentanglementcited by 1
  20. 2026Quality over Quantity: Demonstration Curation via Influence Functions for Data-Centric Robot Learningcited by 1
  21. 2026Training data attribution in diffusion models via mirrored unlearning and noise-consistent skewcited by 1
  22. 2026What's a Credit Worth? A Market Framework for Attribution-Aware Compensation in Generative Musiccited by 1
  23. 2025Concept Reachability in Diffusion Models: Beyond Dataset Constraintscited by 0
  24. 2025Harnessing the Power of Training-Free Techniques in Text-to-2D Generation for Text-to-3D Generation via Score Distillation Samplingcited by 0
  25. 2026SurrogateSHAP: Training-Free Contributor Attribution for Text-to-Image (T2I) Modelscited by 0
  26. 2026Position: Preventing AI-Generated CSAM Necessitates New Approaches to AI Safetycited by 0
  27. 2026FrED: External Data Influence Estimation via Domain Knowledge Graph Groundingcited by 0
  28. 2026Tracing Generated Samples to Training-Data Clusters in Flow-Matching Modelscited by 0
  29. 2026Parallelism, critical windows, and separations among diffusion language modelscited by 0
  30. 2026Distilling Diffusion Score Discrepancy for Efficient Training Data Attributioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Nonparametric Data Attribution for Diffusion Models
  2. 2023Intriguing Properties of Data Attribution on Diffusion Models
  3. 2024Tracing the Roots: Leveraging Temporal Dynamics in Diffusion Trajectories for Origin Attribution
  4. 2024Non-Normal Diffusion Models
  5. 2023The Emergence of Reproducibility and Generalizability in Diffusion Models
  6. 2024Consistent Diffusion Meets Tweedie: Training Exact Ambient Diffusion Models with Noisy Data
  7. 2023Training Data Attribution for Diffusion Models
  8. 2025ReTrack: Data Unlearning in Diffusion Models through Redirecting the Denoising Trajectory