mlsanityData to 7 Oct 2026

Inference-Time Intervention: Eliciting Truthful Answers from a Language Model

arXiv 2306.03341 ↗First posted 6 Jun 2023cs.LG cs.AI cs.CLcited by 1,259

Not on the map. Why?

Cited by 114 in LLM alignment / RLHF / preference optimisation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce Inference-Time Intervention (ITI), a technique designed to enhance the "truthfulness" of large language models (LLMs). ITI operates by shifting model activations during inference, following a set of directions across a limited number of attention heads. This intervention significantly improves the performance of LLaMA models on the TruthfulQA benchmark. On an instruction-finetuned LLaMA called Alpaca, ITI improves its truthfulness from 32.5% to 65.1%. We identify a tradeoff between truthfulness and helpfulness and demonstrate how to balance it by tuning the intervention strength. ITI is minimally invasive and computationally inexpensive. Moreover, the technique is data efficient: while approaches like RLHF require extensive annotations, ITI locates truthful directions using only few hundred examples. Our findings suggest that LLMs may have an internal representation of the likelihood of something being true, even as they produce falsehoods on the surface.

Key facts. Cited by 1,259 papers in the map (data through 7 Oct 2026). Builds on 43 of its 161 references that are in the map. First posted on arXiv 6 Jun 2023.

Builds on

43 of 161 references.

References · 43

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  6. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  7. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  8. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  9. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  10. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  11. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  12. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
31 more
  1. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  2. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  3. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  4. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  5. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 1,998
  6. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  7. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  8. 2016Understanding intermediate layers using linear classifier probescited by 1,279
  9. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,234
  10. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  11. 2022Editing Models with Task Arithmeticcited by 1,076
  12. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,046
  13. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  14. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  15. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
  16. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 805
  17. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 793
  18. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
  19. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  20. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 691
  21. 2022Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskcited by 417
  22. 2020GeDi: Generative Discriminator Guided Sequence Generationcited by 377
  23. 2022Self-critiquing models for assisting human evaluatorscited by 372
  24. 2021True Few-Shot Learning with Language Modelscited by 343
  25. 2022Teaching language models to support answers with verified quotescited by 290
  26. 2023Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Modelscited by 237
  27. 2022Extracting Latent Steering Vectors from Pretrained Language Modelscited by 217
  28. 2022Relative representations enable zero-shot latent space communicationcited by 175
  29. 2017Learning to Generate Reviews and Discovering Sentimentcited by 147
  30. 2023Editing Implicit Assumptions in Text-to-Image Diffusion Modelscited by 128
  31. 2020Language Models are Open Knowledge Graphscited by 95

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 52; 2024: 286; 2025: 348; 2026: 573.

1,259 papers.

Sort:

By area:

  1. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  2. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
  3. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  4. 2023Steering Language Models With Activation Engineeringcited by 824
  5. 2023Large Language Modelscited by 774
  6. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 743
  7. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  8. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
  9. 2023Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervisioncited by 445
  10. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 415
  11. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 381
  12. 2023DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Modelscited by 367
38 more
  1. 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 328
  2. 2023Chain-of-Verification Reduces Hallucination in Large Language Modelscited by 317
  3. 2024INSIDE: LLMs' Internal States Retain the Power of Hallucination Detectioncited by 317
  4. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 313
  5. 2024Knowledge Conflicts for LLMs: A Surveycited by 274
  6. 2024A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Modelscited by 266
  7. 2024Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Securitycited by 257
  8. 2023A Survey of Confidence Estimation and Calibration in Large Language Modelscited by 253
  9. 2024Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMscited by 246
  10. 2023Large Language Model Alignment: A Surveycited by 241
  11. 2023In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steeringcited by 240
  12. 2024LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinationscited by 230
  13. 2024Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modificationscited by 220
  14. 2023Fine-tuning Language Models for Factualitycited by 219
  15. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 204
  16. 2025AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoderscited by 199
  17. 2024ReFT: Representation Finetuning for Language Modelscited by 186
  18. 2023AI Deception: A Survey of Examples, Risks, and Potential Solutionscited by 185
  19. 2024A Mechanistic Understanding of Alignment Algorithms: A Case Study on DPO and Toxicitycited by 179
  20. 2024Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMscited by 174
  21. 2023Can LLM-Generated Misinformation Be Detected?cited by 173
  22. 2024Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similaritiescited by 156
  23. 2024The Geometry of Categorical and Hierarchical Concepts in Large Language Modelscited by 151
  24. 2023RAIN: Your Language Models Can Align Themselves without Finetuningcited by 149
  25. 2024Improving Instruction-Following in Language Models through Activation Steeringcited by 149
  26. 2024How do Large Language Models Handle Multilingualism?cited by 148
  27. 2024Large Legal Fictions: Profiling Legal Hallucinations in Large Language Modelscited by 144
  28. 2024Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Mapscited by 140
  29. 2024Analyzing the Generalization and Reliability of Steering Vectorscited by 130
  30. 2025MemOS: A Memory OS for AI Systemcited by 129
  31. 2024BoNBoN Alignment for Large Language Models and the Sweetness of Best-of-n Samplingcited by 120
  32. 2024Do I Know This Entity? Knowledge Awareness and Hallucinations in Language Modelscited by 114
  33. 2024Vaccine: Perturbation-aware Alignment for Large Language Models against Harmful Fine-tuning Attackcited by 113
  34. 2024A Practical Review of Mechanistic Interpretability for Transformer-Based Language Modelscited by 113
  35. 2024Personalized Steering of Large Language Models: Versatile Steering Vectors Through Bi-directional Preference Optimizationcited by 112
  36. 2024Beyond Functional Correctness: Exploring Hallucinations in LLM-Generated Codecited by 106
  37. 2024AtP*: An efficient and scalable method for localizing LLM behaviour to componentscited by 100
  38. 2024ReDeEP: Detecting Hallucination in Retrieval-Augmented Generation via Mechanistic Interpretabilitycited by 100

50 most cited of 1,259 All 1,259 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Enhancing reasoning accuracy in large language models during inference time
  2. 2025FactReasoner: A Probabilistic Approach to Long-Form Factuality Assessment for Large Language Models
  3. 2025Less is More: Improving LLM Reasoning with Minimal Test-Time Intervention
  4. 2024AI-LieDar: Examine the Trade-off Between Utility and Truthfulness in LLM Agents
  5. 2026Illusions of Confidence? Diagnosing LLM Truthfulness via Neighborhood Consistency
  6. 2026ICE: Intervention-Consistent Explanation Evaluation with Statistical Grounding for LLMs
  7. 2026Summarize Before You Speak with ARACH: A Training-Free Inference-Time Plug-In for Enhancing LLMs via Global Attention Reallocation
  8. 2025Sample Smart, Not Hard: Correctness-First Decoding for Better Reasoning in LLMs