mlsanityData to 7 Oct 2026

Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Models

arXiv 2309.15098 ↗First posted 26 Sep 2023cs.CL cs.AI cs.LGcited by 67

Not on the map. Why?

Abstract (arXiv)

We investigate the internal behavior of Transformer-based Large Language Models (LLMs) when they generate factually incorrect text. We propose modeling factual queries as constraint satisfaction problems and use this framework to investigate how the LLM interacts internally with factual constraints. We find a strong positive relationship between the LLM's attention to constraint tokens and the factual accuracy of generations. We curate a suite of 10 datasets containing over 40,000 prompts to study the task of predicting factual errors with the Llama-2 family across all scales (7B, 13B, 70B). We propose SAT Probe, a method probing attention patterns, that can predict factual errors and fine-grained constraint satisfaction, and allow early error identification. The approach and findings take another step towards using the mechanistic understanding of LLMs to enhance their reliability.

Key facts. Cited by 67 papers in the map (data through 7 Oct 2026). Builds on 41 of its 57 references that are in the map. First posted on arXiv 26 Sep 2023.

Builds on

41 of 57 references.

References · 41

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  6. 2020Extracting Training Data from Large Language Modelscited by 2,144
  7. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  8. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  9. 2019Language Models as Knowledge Bases?cited by 1,922
  10. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
  11. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
  12. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
29 more
  1. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  2. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
  3. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 994
  4. 2022In-context Learning and Induction Headscited by 930
  5. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
  6. 2019Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Prunedcited by 844
  7. 2017Selective Classification for Deep Neural Networkscited by 799
  8. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 787
  9. 2022Quantifying Memorization Across Neural Language Modelscited by 755
  10. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
  11. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 579
  12. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 512
  13. 2022Large Language Models Struggle to Learn Long-Tail Knowledgecited by 452
  14. 20218-bit Optimizers via Block-wise Quantizationcited by 395
  15. 2023Finding Neurons in a Haystack: Case Studies with Sparse Probingcited by 328
  16. 2023How Language Model Hallucinations Can Snowballcited by 310
  17. 2023A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generationcited by 202
  18. 2023Linearity of Relation Decoding in Transformer Language Modelscited by 190
  19. 2023Head-to-Tail: How Knowledgeable are Large Language Models (LLMs)? A.K.A. Will LLMs Replace Knowledge Graphs?cited by 167
  20. 2023LM vs LM: Detecting Factual Errors via Cross Examinationcited by 155
  21. 2019Challenges in Building Intelligent Open-domain Dialog Systemscited by 147
  22. 2023Emergent and Predictable Memorization in Large Language Modelscited by 147
  23. 2022Analyzing Transformers in Embedding Spacecited by 131
  24. 2023Self-contradictory Hallucinations of Large Language Models: Evaluation, Detection and Mitigationcited by 123
  25. 2023AI Transparency in the Age of LLMs: A Human-Centered Research Roadmapcited by 119
  26. 2023Scan and Snap: Understanding Training Dynamics and Token Composition in 1-layer Transformercited by 101
  27. 2019Do Attention Heads in BERT Track Syntactic Dependencies?cited by 87
  28. 2023Overthinking the Truth: Understanding how Language Models Process False Demonstrationscited by 78
  29. 2023Beyond Confidence: Reliable Models Should Also Consider Atypicalitycited by 25

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 29; 2025: 30; 2026: 6.

67 papers.

Sort:

By area:

  1. 2024TextGrad: Automatic "Differentiation" via Textcited by 250
  2. 2024LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinationscited by 230
  3. 2024Devils in Middle Layers of Large Vision-Language Models: Interpreting, Detecting and Mitigating Object Hallucinations via Attention Lenscited by 139
  4. 2024Do I Know This Entity? Knowledge Awareness and Hallucinations in Language Modelscited by 114
  5. 2024The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language Modelscited by 87
  6. 2024Small Language Models Need Strong Verifiers to Self-Correct Reasoningcited by 85
  7. 2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 83
  8. 2023Do Androids Know They're Only Dreaming of Electric Sheep?cited by 67
  9. 2024Knowledge Circuits in Pretrained Transformerscited by 60
  10. 2024Understanding Information Storage and Transfer in Multi-modal Large Language Modelscited by 48
  11. 2024In-Context Sharpness as Alerts: An Inner Representation Perspective for Hallucination Mitigationcited by 43
  12. 2024On Large Language Models' Hallucination with Regard to Known Factscited by 43
38 more
  1. 2024Estimating Knowledge in Large Language Models Without Generating a Single Tokencited by 39
  2. 2024Mechanistic Understanding and Mitigation of Language Model Non-Factual Hallucinationscited by 38
  3. 2024Understanding Finetuning for Factual Knowledge Extractioncited by 31
  4. 2024RAG and RAU: A Survey on Retrieval-Augmented Language Model in Natural Language Processingcited by 25
  5. 2024DeCoRe: Decoding by Contrasting Retrieval Heads to Mitigate Hallucinationscited by 23
  6. 2025A Head to Predict and a Head to Question: Pre-trained Uncertainty Quantification Heads for Hallucination Detection in LLM Outputscited by 20
  7. 2024Constructing Benchmarks and Interventions for Combating Hallucinations in LLMscited by 19
  8. 2025Robust Hallucination Detection in LLMs via Adaptive Token Selectioncited by 19
  9. 2024Eureka: Evaluating and Understanding Large Foundation Modelscited by 18
  10. 2025SelfElicit: Your Language Model Secretly Knows Where is the Relevant Evidencecited by 18
  11. 2025Reasoning Models Hallucinate More: Factuality-Aware Reinforcement Learning for Large Reasoning Modelscited by 18
  12. 2024Prompt Perturbation in Retrieval-Augmented Generation based Large Language Modelscited by 17
  13. 2025Correctness Assessment of Code Generated by Large Language Models Using Internal Representationscited by 17
  14. 2025Probing for Arithmetic Errors in Language Modelscited by 17
  15. 2025Attention-guided Self-reflection for Zero-shot Hallucination Detection in Large Language Modelscited by 16
  16. 2025Beyond Token Probes: Hallucination Detection via Activation Tensors with ACT-ViTcited by 16
  17. 2024Anchored Answers: Unravelling Positional Bias in GPT-2's Multiple-Choice Questionscited by 15
  18. 2024Distinguishing Ignorance from Error in LLM Hallucinationscited by 14
  19. 2025Efficient Hallucination Detection for LLMs Using Uncertainty-Aware Attention Headscited by 14
  20. 2025Attributing Response to Context: A Jensen-Shannon Divergence Driven Mechanistic Study of Context Attribution in Retrieval-Augmented Generationcited by 12
  21. 2025LLM Microscope: What Model Internals Reveal About Answer Correctness and Context Utilizationcited by 11
  22. 2023KITAB: Evaluating LLMs on Constraint Satisfaction for Information Retrievalcited by 10
  23. 2025Mass-Editing Memory with Attention in Transformers: A cross-lingual exploration of knowledgecited by 10
  24. 2025Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safetycited by 10
  25. 2024Whispers that Shake Foundations: Analyzing and Mitigating False Premise Hallucinations in Large Language Modelscited by 9
  26. 2024Zero-resource Hallucination Detection for Text Generation via Graph-based Contextual Knowledge Triples Modelingcited by 9
  27. 2025Language Models Can Predict Their Own Behaviorcited by 8
  28. 2025A Single Direction of Truth: An Observer Model's Linear Residual Probe Exposes and Steers Contextual Hallucinationscited by 8
  29. 2025Tracing the Traces: Latent Temporal Signals for Efficient and Accurate Reasoningcited by 8
  30. 2025Do LLMs Really Know What They Don't Know? Internal States Mainly Reflect Knowledge Recall Rather Than Truthfulnesscited by 7
  31. 2024Attention Speaks Volumes: Localizing and Mitigating Bias in Language Modelscited by 6
  32. 2024BenchAgents: Multi-Agent Systems for Structured Benchmark Creationcited by 6
  33. 2025When Do LLMs Admit Their Mistakes? Understanding The Role Of Model Belief In Retractioncited by 6
  34. 2025Popular but Wrong: Understanding and Mitigating LLM Overconfidence through Knowledge Popularitycited by 6
  35. 2025metaTextGrad: Automatically optimizing language model optimizerscited by 6
  36. 2025Evaluating and Improving Robustness in Large Language Models: A Survey and Future Directionscited by 5
  37. 2025Neural Message-Passing on Attention Graphs for Hallucination Detectioncited by 5
  38. 2025Cross-Layer Attention Probing for Fine-Grained Hallucination Detectioncited by 4

50 most cited of 67 All 67 (JSON)