mlsanityData to 14 Oct 2026

OLMoTrace: Tracing Language Model Outputs Back to Trillions of Training Tokens

arXiv 2504.07096 ↗First posted 9 Apr 2025cs.CLcited by 27

Not on the map. Why?

Abstract (arXiv)

We present OLMoTrace, the first system that traces the outputs of language models back to their full, multi-trillion-token training data in real time. OLMoTrace finds and shows verbatim matches between segments of language model output and documents in the training text corpora. Powered by an extended version of infini-gram (Liu et al., 2024), our system returns tracing results within a few seconds. OLMoTrace can help users understand the behavior of language models through the lens of their training data. We showcase how it can be used to explore fact checking, hallucination, and the creativity of language models. OLMoTrace is publicly available and fully open-source.

Key facts. Cited by 27 papers in the map (data through 14 Oct 2026). Builds on 7 of its 11 references that are in the map. First posted on arXiv 9 Apr 2025.

Builds on

7 of 11 references.

References · 7

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  2. 2017Understanding Black-box Predictions via Influence Functionscited by 1,741
  3. 2024OLMoE: Open Mixture-of-Experts Language Modelscited by 301
  4. 2024Infini-gram: Scaling Unbounded n-gram Language Models to a Trillion Tokenscited by 153
  5. 2020Explaining Black Box Predictions and Unveiling Data Artifacts through Influence Functionscited by 144
  6. 2024Training Language Models to Generate Text with Citations via Fine-grained Rewardscited by 42
  7. 2022ORCA: Interpreting Prompted Language Models via Locating Supporting Data Evidence in the Ocean of Pretraining Datacited by 34

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 11; 2026: 16.

27 papers.

Sort:

By area:

  1. 2025Infini-gram mini: Exact n-gram Search at the Internet Scale with FM-Indexcited by 14
  2. 2025QuCo-RAG: Quantifying Uncertainty from the Pre-training Corpus for Dynamic Retrieval-Augmented Generationcited by 12
  3. 2026Gemma Needs Help: Investigating and Mitigating Emotional Instability in LLMscited by 7
  4. 2025Measuring LLM Novelty As The Frontier Of Original And High-Quality Outputcited by 6
  5. 2025"Lost-in-the-Later": Framework for Quantifying Contextual Grounding in Large Language Modelscited by 5
  6. 2025DATE-LM: Benchmarking Data Attribution Evaluation for Large Language Modelscited by 5
  7. 2026Overstating Attitudes, Ignoring Networks: LLM Biases in Simulating Misinformation Susceptibilitycited by 5
  8. 2025One Model, Many Morals: Uncovering Cross-Linguistic Misalignments in Computational Moral Reasoningcited by 4
  9. 2026Influential Training Data Retrieval for Explaining Verbalized Confidence of LLMscited by 4
  10. 2025ExPLAIND: Unifying Model, Data, and Training Attribution to Study Model Behaviorcited by 3
  11. 2026Tracing the Data Trail: A Survey of Data Provenance, Transparency and Traceability in LLMscited by 3
  12. 2025Cite Pretrain: Retrieval-Free Knowledge Attribution for Large Language Modelscited by 2
15 more
  1. 2025Revisiting the Past: Data Unlearning with Model State Historycited by 2
  2. 2026NanoKnow: How to Know What Your Language Model Knowscited by 2
  3. 2026How Open Must Language Models be to Enable Reliable Scientific Inference?cited by 2
  4. 2026Generative AI floods and dilutes the market for bookscited by 2
  5. 2026A Bayesian Information-Theoretic Approach to Data Attributioncited by 1
  6. 2026Why are all LLMs Obsessed with Japanese Culture? On the Hidden Cultural and Regional Biases of LLMscited by 1
  7. 2026SuperValid: Capability-Aligned OOD Validation for Generalizable Downstream Scalingcited by 1
  8. 2026LLMs Can Leak Training Data But Do They Want To? A Propensity-Aware Evaluation of Memorization in LLMscited by 1
  9. 2025LLM generation novelty through the lens of semantic similaritycited by 0
  10. 2025TrackList: Tracing Back Query Linguistic Diversity for Head and Tail Knowledge in Open Large Language Modelscited by 0
  11. 2026Feature Starvation as Geometric Instability in Sparse Autoencoderscited by 0
  12. 2026Efficient and Scalable Provenance Tracking for LLM-Generated Code Snippetscited by 0
  13. 2026Quantifying the Agreement Between Data-Influence and Data-Similarity to Understand LLM Behaviorcited by 0
  14. 2026Tracing LLM Behavior to the Training Data with Empirical Next-Token Distributionscited by 0
  15. 2026What to Forget in Unlearning? Forget Set Curation for Language Modelscited by 0