mlsanityData to 7 Oct 2026

ContextCite: Attributing Model Generation to Context

arXiv 2409.00729 ↗First posted 1 Sep 2024cs.LG cs.CLcited by 93

Not on the map. Why?

Abstract (arXiv)

How do language models use information provided as context when generating a response? Can we infer whether a particular generated statement is actually grounded in the context, a misinterpretation, or fabricated? To help answer these questions, we introduce the problem of context attribution: pinpointing the parts of the context (if any) that led a model to generate a particular statement. We then present ContextCite, a simple and scalable method for context attribution that can be applied on top of any existing language model. Finally, we showcase the utility of ContextCite through three applications: (1) helping verify generated statements (2) improving response quality by pruning the context and (3) detecting poisoning attacks. We provide code for ContextCite at https://github.com/MadryLab/context-cite.

Key facts. Cited by 93 papers in the map (data through 7 Oct 2026). Builds on 56 of its 74 references that are in the map. First posted on arXiv 1 Sep 2024.

Builds on

56 of 74 references.

References · 56

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  4. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  5. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  6. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  7. 2023Mistral 7Bcited by 3,065
  8. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  9. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  10. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  11. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  12. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
38 more
  1. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  2. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  3. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  4. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  5. 2017SmoothGrad: removing noise by adding noisecited by 1,097
  6. 2019Attention is not Explanationcited by 916
  7. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 883
  8. 2016Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyondcited by 809
  9. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
  10. 2020Quantifying Attention Flow in Transformerscited by 759
  11. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
  12. 2023Enabling Large Language Models to Generate Text with Citationscited by 584
  13. 2020TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languagescited by 486
  14. 2019Is Attention Interpretable?cited by 387
  15. 2023Studying Large Language Model Generalization with Influence Functionscited by 311
  16. 2023Evaluating Verifiability in Generative Search Enginescited by 297
  17. 2016Not Just a Black Box: Learning Important Features Through Propagating Activation Differencescited by 295
  18. 2022Teaching language models to support answers with verified quotescited by 290
  19. 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 286
  20. 2023TRAK: Attributing Model Behavior at Scalecited by 273
  21. 2021Measuring Attribution in Natural Language Generation Modelscited by 253
  22. 2023FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarioscited by 247
  23. 2020Explaining by Removing: A Unified Framework for Model Explanationcited by 218
  24. 2015Visualizing and Understanding Neural Models in NLPcited by 199
  25. 2022Datamodels: Predicting Predictions from Training Datacited by 186
  26. 2023DataInf: Efficiently Estimating Data Influence in LoRA-tuned LLMs and Diffusion Modelscited by 121
  27. 2023Calibrated Language Models Must Hallucinatecited by 114
  28. 2023Complex Claim Verification with Evidence Retrieved in the Wildcited by 110
  29. 2022Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Modelscited by 108
  30. 2024DsDm: Model-Aware Dataset Selection with Datamodelscited by 103
  31. 2024AtP*: An efficient and scalable method for localizing LLM behaviour to componentscited by 100
  32. 2023Attention Sorting Combats Recency Bias In Long Context Language Modelscited by 89
  33. 2024Neural Exec: Learning (and Learning from) Execution Triggers for Prompt Injection Attackscited by 78
  34. 2023In-context Example Selection with Influencescited by 60
  35. 2022Data Curation Alone Can Stabilize In-context Learningcited by 58
  36. 2023PURR: Efficiently Editing Language Model Hallucinations by Denoising Language Model Corruptionscited by 53
  37. 2023Sequential Integrated Gradients: a simple but effective method for explaining language modelscited by 52
  38. 2023Effective Large Language Model Adaptation for Improved Grounding and Citation Generationcited by 44

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 8; 2025: 46; 2026: 39.

93 papers.

Sort:

By area:

  1. 2025Design Patterns for Securing LLM Agents against Prompt Injectionscited by 67
  2. 2024Steering Away from Harm: An Adaptive Approach to Defending Vision Language Model Against Jailbreakscited by 44
  3. 2024AttriBoT: A Bag of Tricks for Efficiently Approximating Leave-One-Out Context Attributioncited by 23
  4. 2024Permissive Information-Flow Analysis for Large Language Modelscited by 22
  5. 2025SelfCite: Self-Supervised Alignment for Context Attribution in Large Language Modelscited by 19
  6. 2025ProxySPEX: Inference-Efficient Interpretability via Sparse Feature Interactions in LLMscited by 16
  7. 2025TokenShapley: Token Level Context Attribution with Shapley Valuecited by 14
  8. 2025LAQuer: Localized Attribution Queries in Content-grounded Generationcited by 13
  9. 2026Spurious Rewards Paradox: Mechanistically Understanding How RLVR Activates Memorization Shortcuts in LLMscited by 13
  10. 2026CausalArmor: Efficient Indirect Prompt Injection Guardrails via Causal Attributioncited by 13
  11. 2024LitLLMs, LLMs for Literature Review: Are we there yet?cited by 12
  12. 2025SPEX: Scaling Feature Interaction Explanations for LLMscited by 12
38 more
  1. 2025Attributing Response to Context: A Jensen-Shannon Divergence Driven Mechanistic Study of Context Attribution in Retrieval-Augmented Generationcited by 12
  2. 2025Understanding and Leveraging the Expert Specialization of Context Faithfulness in Mixture-of-Experts LLMscited by 12
  3. 2025Who Taught the Lie? Responsibility Attribution for Poisoned Knowledge in Retrieval-Augmented Generationcited by 12
  4. 2025Learning to Attribute with Attentioncited by 11
  5. 2025The Atlas of In-Context Learning: How Attention Heads Shape In-Context Retrieval Augmentationcited by 11
  6. 2025TracLLM: A Generic Framework for Attributing Long Context LLMscited by 11
  7. 2025Resisting Contextual Interference in RAG via Parametric-Knowledge Reinforcementcited by 11
  8. 2025Can LLMs Explain Themselves Counterfactually?cited by 10
  9. 2025Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safetycited by 10
  10. 2025HoT: Highlighted Chain of Thought for Referencing Supporting Facts from Inputscited by 9
  11. 2025The Role of Sparsity for Length Generalization in Transformerscited by 8
  12. 2026The Why Behind the Action: Unveiling Internal Drivers via Agentic Attributioncited by 8
  13. 2025Source Attribution in Retrieval-Augmented Generationcited by 7
  14. 2025Linguistic Nepotism: Trading-off Quality for Language Preference in Multilingual RAGcited by 7
  15. 2024Estimating Privacy Leakage of Augmented Contextual Knowledge in Language Modelscited by 6
  16. 2024Attention with Dependency Parsing Augmentation for Fine-Grained Attributioncited by 5
  17. 2025MedPAIR: Measuring Physicians and AI Relevance Alignment in Medical Question Answeringcited by 5
  18. 2025AttnTrace: Contextual Attribution of Prompt Injection and Knowledge Corruptioncited by 5
  19. 2025B-cos LM: Efficiently Transforming Pre-trained Language Models for Improved Explainabilitycited by 4
  20. 2025Transparentize the Internal and External Knowledge Utilization in LLMs with Trustworthy Citationcited by 4
  21. 2025ConSens: Assessing context grounding in open-book question answeringcited by 4
  22. 2025Unsupervised Word-level Quality Estimation for Machine Translation Through the Lens of Annotators (Dis)agreementcited by 4
  23. 2025GenerationPrograms: Fine-grained Attribution with Executable Programscited by 4
  24. 2026Beware What You Autocomplete: Forensic Attribution of Backdoored Code Completionscited by 4
  25. 2024Time-Reversal Provides Unsupervised Feedback to LLMscited by 3
  26. 2025HealthContradict: Evaluating Biomedical Knowledge Conflicts in Language Modelscited by 3
  27. 2025MaxShapley: Towards Incentive-compatible Generative Search with Fair Context Attributioncited by 3
  28. 2025Explaining the Reasoning of Large Language Models Using Attribution Graphscited by 3
  29. 2026AttriMem: Attribution-Guided Process Feedback for Agent Memory Constructioncited by 3
  30. 2024A Context-aware Framework for Translation-mediated Conversationscited by 2
  31. 2025On Synthesizing Data for Context Attribution in Question Answeringcited by 2
  32. 2025Know Or Not: a library for evaluating out-of-knowledge base robustnesscited by 2
  33. 2025Context Attribution with Multi-Armed Bandit Optimizationcited by 2
  34. 2026What if AI systems weren't chatbots?cited by 2
  35. 2025Multimedia and Visual Analytics in the Agentic Eracited by 1
  36. 2025Is Active Persona Inference Necessary for Aligning Small Models to Personal Preferences?cited by 1
  37. 2025DePass: Unified Feature Attributing by Simple Decomposed Forward Passcited by 1
  38. 2025What Really Counts? Examining Step and Token Level Attribution in Multilingual CoT Reasoningcited by 1

50 most cited of 93 All 93 (JSON)