mlsanityData to 14 Oct 2026

Tracing the Data Trail: A Survey of Data Provenance, Transparency and Traceability in LLMs

arXiv 2601.14311 ↗First posted 19 Jan 2026cs.CR cs.AI cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) are deployed at scale, yet their training data life cycle remains opaque. This survey synthesizes research from the past ten years on three tightly coupled axes: (1) data provenance, (2) transparency, and (3) traceability, and three supporting pillars: (4) bias & uncertainty, (5) data privacy, and (6) tools and techniques that operationalize them. A central contribution is a proposed taxonomy defining the field's domains and listing corresponding artifacts. Through analysis of 95 publications, this work identifies key methodologies concerning data generation, watermarking, bias measurement, data curation, data privacy, and the inherent trade-off between transparency and opacity.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 51 of its 129 references that are in the map. First posted on arXiv 19 Jan 2026.

Builds on

51 of 129 references.

References · 51

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
  3. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,173
  4. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,160
  5. 2022Toy Models of Superpositioncited by 971
  6. 2024BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domainscited by 301
  7. 2021Measuring Attribution in Natural Language Generation Modelscited by 256
  8. 2023Large Language Models are Versatile Decomposers: Decompose Evidence and Questions for Table-based Reasoningcited by 178
  9. 2023Osprey: Pixel Understanding with Visual Instruction Tuningcited by 159
  10. 2023Decomposing Uncertainty for Large Language Models through Input Clarification Ensemblingcited by 114
  11. 2023MentaLLaMA: Interpretable Mental Health Analysis on Social Media with Large Language Modelscited by 63
  12. 2025How much do language models memorize?cited by 61
38 more
  1. 2022Data Feedback Loops: Model-driven Amplification of Dataset Biasescited by 60
  2. 2024LLMmap: Fingerprinting For Large Language Modelscited by 58
  3. 2022Multi-LexSum: Real-World Summaries of Civil Rights Lawsuits at Multiple Granularitiescited by 47
  4. 2024TuneTables: Context Optimization for Scalable Prior-Data Fitted Networkscited by 47
  5. 2024Authorship Attribution in the Era of LLMs: Problems, Methodologies, and Challengescited by 47
  6. 2023Interpretable Online Log Analysis Using Large Language Models with Prompt Strategiescited by 38
  7. 2024BMRetriever: Tuning Large Language Models as Better Biomedical Text Retrieverscited by 38
  8. 2024ReposVul: A Repository-Level High-Quality Vulnerability Datasetcited by 36
  9. 2025OLMoTrace: Tracing Language Model Outputs Back to Trillions of Training Tokenscited by 27
  10. 2022The Effects of Data Quality on Machine Learning Performance on Tabular Datacited by 19
  11. 2025From Aleatoric to Epistemic: Exploring Uncertainty Quantification Techniques in Artificial Intelligencecited by 18
  12. 2024The Mirrored Influence Hypothesis: Efficient Data Influence Estimation by Harnessing Forward Passescited by 16
  13. 2024Assessing and Verifying Task Utility in LLM-Powered Applicationscited by 16
  14. 2025Understanding Silent Data Corruption in LLM Trainingcited by 15
  15. 2023Benchmarking Causal Study to Interpret Large Language Models for Source Codecited by 14
  16. 2024Advancing Large Language Model Attribution through Self-Improvingcited by 14
  17. 2025Enhancing Semantic Consistency of Large Language Models through Model Editing: An Interpretability-Oriented Approachcited by 14
  18. 2020Towards Interpretable Deep Learning Models for Knowledge Tracingcited by 13
  19. 2024MetaToken: Detecting Hallucination in Image Descriptions by Meta Classificationcited by 13
  20. 2023HiPrompt: Few-Shot Biomedical Knowledge Fusion via Hierarchy-Oriented Promptingcited by 11
  21. 2024GPTs Are Multilingual Annotators for Sequence Generation Taskscited by 11
  22. 2021Contextual Semantic Parsing for Multilingual Task-Oriented Dialoguescited by 10
  23. 2024FuxiTranyu: A Multilingual Large Language Model Trained with Balanced Datacited by 10
  24. 2023InstructIE: A Bilingual Instruction-based Information Extraction Datasetcited by 9
  25. 2025CarMem: Enhancing Long-Term Memory in LLM Voice Assistants through Category-Boundingcited by 9
  26. 2023Discriminating Human-authored from ChatGPT-Generated Code Via Discernable Feature Analysiscited by 7
  27. 2024On Efficient Language and Vision Assistants for Visually-Situated Natural Language Understanding: What Matters in Reading and Reasoningcited by 7
  28. 2024Does Multiple Choice Have a Future in the Age of Generative AI? A Posttest-only RCTcited by 7
  29. 2024An Integrated Data Processing Framework for Pretraining Foundation Modelscited by 6
  30. 2024Efficient Data Generation for Source-grounded Information-seeking Dialogs: A Use Case for Meeting Transcriptscited by 6
  31. 2018An unsupervised and customizable misspelling generator for mining noisy health-related text sourcescited by 5
  32. 2023Generative AI in Writing Research Papers: A New Type of Algorithmic Bias and Uncertainty in Scholarly Workcited by 5
  33. 2025Large Language Model Sourcing: A Surveycited by 5
  34. 2024Explainable CTR Prediction via LLM Reasoningcited by 4
  35. 2023Studying Logging Practice in Machine Learning-based Applicationscited by 3
  36. 2024Can Large Language Model Predict Employee Attrition?cited by 3
  37. 2023Distantly-Supervised Joint Extraction with Noise-Robust Learningcited by 2
  38. 2024Topic Modelling: Going Beyond Token Outputscited by 1

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026From GPT-3 to GPT-5: Mapping their capabilities, scope, limitations, and consequencescited by 3
  2. 2026RLVR Datasets and Where to Find Them: Tracing Data Lineage for Better Training Datacited by 1
  3. 2026Detecting Pretraining Data in Large Language Models from a Free-Energy Perspectivecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026LLM Watermarking as Big Data Provenance: A Deployment-Oriented Systematization
  2. 2026Implicit Identity Technologies for LLMs: Fingerprinting and Watermarking across Datasets, Models, and Generated Content
  3. 2025A Statistical Hypothesis Testing Framework for Data Misappropriation Detection in Large Language Models
  4. 2026A Survey on Evaluating Quality and Trustworthiness in LLM-Generated Data
  5. 2024Generated Data with Fake Privacy: Hidden Dangers of Fine-tuning Large Language Models on Generated Data
  6. 2026Auditing Data Provenance in LLM Fine-tuning via Intrinsic Distributional Fingerprints
  7. 2024Purifying Large Language Models by Ensembling a Small Language Model
  8. 2025A Survey on Privacy Risks and Protection in Large Language Models