mlsanityData to 7 Oct 2026

Towards User-Focused Research in Training Data Attribution for Human-Centered Explainable AI

arXiv 2409.16978 ↗First posted 25 Sep 2024cs.HC cs.AI cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

Explainable AI (XAI) aims to make AI systems more transparent, yet many practices emphasise mathematical rigour over practical user needs. We propose an alternative to this model-centric approach by following a design thinking process for the emerging XAI field of training data attribution (TDA), which risks repeating solutionist patterns seen in other subfields. However, because TDA is in its early stages, there is a valuable opportunity to shape its direction through user-centred practices. We engage directly with machine learning developers via a needfinding interview study (N=6) and a scenario-based interactive user study (N=31) to ground explanations in real workflows. Our exploration of the TDA design space reveals novel tasks for data-centric explanations useful to developers, such as grouping training samples behind specific model behaviours or identifying undersampled data. We invite the TDA, XAI, and HCI communities to engage with these tasks to strengthen their research's practical relevance and human impact.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 46 of its 96 references that are in the map. First posted on arXiv 25 Sep 2024.

Builds on

46 of 96 references.

References · 46

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  4. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  5. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  6. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  7. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  8. 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
  9. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,400
  10. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,201
  11. 2018Sanity Checks for Saliency Mapscited by 1,149
  12. 2017SmoothGrad: removing noise by adding noisecited by 1,097
34 more
  1. 2019Data Shapley: Equitable Valuation of Data for Machine Learningcited by 617
  2. 2020Estimating Training Data Influence by Tracing Gradient Descentcited by 565
  3. 2020What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimationcited by 492
  4. 2023Studying Large Language Model Generalization with Influence Functionscited by 311
  5. 2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AIcited by 299
  6. 2023TRAK: Attributing Model Behavior at Scalecited by 273
  7. 2020Questioning the AI: Informing Design Practices for Explainable AI User Experiencescited by 251
  8. 2020Influence Functions in Deep Learning Are Fragilecited by 247
  9. 2022Datamodels: Predicting Predictions from Training Datacited by 186
  10. 2019On the Accuracy of Influence Functions for Measuring Group Effectscited by 146
  11. 2022If Influence Functions are the Answer, Then What is the Question?cited by 142
  12. 2022Training Data Influence Analysis and Estimation: A Surveycited by 134
  13. 2021Expanding Explainability: Towards Social Transparency in AI systemscited by 124
  14. 2021HIVE: Evaluating the Human Interpretability of Visual Explanationscited by 105
  15. 2020Human-centered Explainable AI: Towards a Reflective Sociotechnical Approachcited by 90
  16. 2024What is Your Data Worth to GPT? LLM-Scale Data Valuation with Influence Functionscited by 88
  17. 2022Towards Human-centered Explainable AI: A Survey of User Studies for Model Explanationscited by 84
  18. 2019On Second-Order Group Influence Functions for Black-Box Predictionscited by 78
  19. 2021Input Similarity from the Neural Network Perspectivecited by 78
  20. 2020Human Factors in Model Interpretability: Industry Practices, Challenges, and Needscited by 73
  21. 2019Data Cleansing for Models Trained with SGDcited by 67
  22. 2023In Search of Verifiability: Explanations Rarely Enable Complementary Performance in AI-Advised Decision Makingcited by 50
  23. 2021Interactive Label Cleaning with Example-based Explanationscited by 48
  24. 2021The Who in XAI: How AI Background Shapes Perceptions of AI Explanationscited by 48
  25. 2023Intriguing Properties of Data Attribution on Diffusion Modelscited by 47
  26. 2021Combining Feature and Instance Attribution to Detect Artifactscited by 38
  27. 2022Visual correspondence-based explanations improve AI robustness and human-AI team accuracycited by 37
  28. 2024Training Data Attribution via Approximate Unrolled Differentiationcited by 33
  29. 2023Revisiting the Fragility of Influence Functionscited by 20
  30. 2023Error Discovery by Clustering Influence Embeddingscited by 7
  31. 2023Users are the North Star for AI Transparencycited by 5
  32. 2024Data Debiasing with Datamodels (D3M): Improving Subgroup Robustness via Data Selectioncited by 5
  33. 2023Invisible Users: Uncovering End-Users' Requirements for Explainable AI via Explanation Forms and Goalscited by 4
  34. 2024Dissecting users' needs for search result explanationscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 1.

3 papers.

  1. 2024Scalable Influence and Fact Tracing for Large Language Model Pretrainingcited by 40
  2. 2025Better Training Data Attribution via Better Inverse Hessian-Vector Productscited by 10
  3. 2026Attributing Cohen's d: Training Data Attribution for Disease-Related Effects in Normative Age Biomarkerscited by 0