mlsanityData to 7 Oct 2026

Adaptive Activation Steering: A Tuning-Free LLM Truthfulness Improvement Method for Diverse Hallucinations Categories

arXiv 2406.00034 ↗First posted 26 May 2024cs.CL cs.AIcited by 74

Not on the map. Why?

Abstract (arXiv)

Recent studies have indicated that Large Language Models (LLMs) harbor an inherent understanding of truthfulness, yet often fail to consistently express it and generate false statements. This gap between "knowing" and "telling" poses a challenge for ensuring the truthfulness of generated content. Inspired by recent work on the practice of encoding human-interpretable concepts linearly within large language models, we treat truthfulness as a specially linearly encoded concept within LLMs, and introduce Adaptive Activation Steering (ACT), a tuning-free method that adaptively shifts LLM's activations in the "truthful" direction during inference. ACT addresses diverse categories of hallucinations by utilizing diverse truthfulness-related steering vectors and adjusting the steering intensity adaptively. Applied as an add-on across various models, ACT significantly improves truthfulness in LLaMA (uparrow 142%), LLaMA2 (uparrow 24%), Alpaca (uparrow 36%), Vicuna (uparrow 28%), LLaMA2-Chat (uparrow 19%), and LLaMA3(uparrow 34%). Furthermore, we verify ACT's scalability across larger models (13B, 33B, 65B), underscoring the adaptability of ACT to large-scale language models. Our code is available at https://github.com/tianlwang/ACT.

Key facts. Cited by 74 papers in the map (data through 7 Oct 2026). Builds on 39 of its 58 references that are in the map. First posted on arXiv 26 May 2024.

Builds on

39 of 58 references.

References · 39

  1. 2017Attention Is All You Needcited by 59,932
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  7. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  8. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  9. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  10. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  11. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  12. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
27 more
  1. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  2. 2016Understanding intermediate layers using linear classifier probescited by 1,279
  3. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  4. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  5. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
  6. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,046
  7. 2022Toy Models of Superpositioncited by 928
  8. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
  9. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 793
  10. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
  11. 2019Interpreting the Latent Space of GANs for Semantic Face Editingcited by 691
  12. 2015Sequence Level Training with Recurrent Neural Networkscited by 568
  13. 2023Erasing Concepts from Diffusion Modelscited by 530
  14. 2015Autoencoding beyond pixels using a learned similarity metriccited by 407
  15. 2023Chain-of-Verification Reduces Hallucination in Large Language Modelscited by 317
  16. 2022Teaching language models to support answers with verified quotescited by 290
  17. 2020Understanding the Role of Individual Units in a Deep Neural Networkcited by 254
  18. 2023LEACE: Perfect linear concept erasure in closed formcited by 254
  19. 2022Extracting Latent Steering Vectors from Pretrained Language Modelscited by 217
  20. 2020Semantic Photo Manipulation with a Generative Image Priorcited by 178
  21. 2021EditGAN: High-Precision Semantic Image Editingcited by 156
  22. 2023A Critical Evaluation of Evaluations for Long-form Question Answeringcited by 111
  23. 2016Deep Feature Interpolation for Image Content Changescited by 80
  24. 2023Improving Activation Steering in Language Models with Mean-Centringcited by 64
  25. 2024Explainable Fake News Detection With Large Language Model via Defense Among Competing Wisdomcited by 49
  26. 2023Personas as a Way to Model Truthfulness in Language Modelscited by 46
  27. 2023Efficient fair PCA for fair representation learningcited by 23

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 41; 2026: 29.

74 papers.

Sort:

By area:

  1. 2024Programming Refusal with Conditional Activation Steeringcited by 191
  2. 2025MemOS: A Memory OS for AI Systemcited by 129
  3. 2025SAFE: Multitask Failure Detection for Vision-Language-Action Modelscited by 74
  4. 2025MemOS: An Operating System for Memory-Augmented Generation (MAG) in Large Language Modelscited by 68
  5. 2024SCANS: Mitigating the Exaggerated Safety for LLMs via Safety-Conscious Activation Steeringcited by 45
  6. 2025Angular Steering: Behavior Control via Rotation in Activation Spacecited by 45
  7. 2025A Unified Understanding and Evaluation of Steering Methodscited by 40
  8. 2025AdaSteer: Your Aligned LLM is Inherently an Adaptive Jailbreak Defendercited by 37
  9. 2025SAE-SSV: Supervised Steering in Sparse Representation Spaces for Reliable Control of Language Modelscited by 28
  10. 2025To Steer or Not to Steer? Mechanistic Error Reduction with Abstention for Language Modelscited by 26
  11. 2024Beyond Single Concept Vector: Modeling Concept Subspace in LLMs with Gaussian Distributioncited by 24
  12. 2025Representation Engineering for Large-Language Models: Survey and Research Challengescited by 18
38 more
  1. 2025Understanding and Mitigating Overrefusal in LLMs from an Unveiling Perspective of Safety Decision Boundarycited by 17
  2. 2026Spherical Steering: Geometry-Aware Activation Rotation for Language Modelscited by 17
  3. 2025Adaptive Multi-Subspace Representation Steering for Attribute Alignment in Large Language Modelscited by 16
  4. 2025Improving Multilingual Language Models by Aligning Representations through Steeringcited by 11
  5. 2025PIXEL: Adaptive Steering Via Position-wise Injection with eXact Estimated Levels under Subspace Calibrationcited by 10
  6. 2025The Geometries of Truth Are Orthogonal Across Taskscited by 9
  7. 2025Does higher interpretability imply better utility? A Pairwise Analysis on Sparse Autoencoderscited by 9
  8. 2025MLLMEraser: Achieving Test-Time Unlearning in Multimodal Large Language Models through Activation Steeringcited by 9
  9. 2025LLM Knowledge is Brittle: Truthfulness Representations Rely on Superficial Resemblancecited by 9
  10. 2025Incentivizing Truthful Language Models via Peer Elicitation Gamescited by 8
  11. 2025Guiding Giants: Lightweight Controllers for Weighted Activation Steering in LLMscited by 8
  12. 2025KV Cache Steering for Controlling Frozen LLMscited by 8
  13. 2025SteerVLM: Robust Model Control through Lightweight Activation Steering for Vision Language Modelscited by 8
  14. 2025Effectively Steer LLM To Follow Preference via Building Confident Directionscited by 7
  15. 2025Model Directions, Not Words: Mechanistic Topic Models Using Sparse Autoencoderscited by 7
  16. 2025Steering When Necessary: Flexible Steering Large Language Models with Backtrackingcited by 7
  17. 2026Efficient and accurate steering of Large Language Models through attention-guided feature learningcited by 6
  18. 2026Spectral Attention Steering for Prompt Highlightingcited by 6
  19. 2025Magical: Medical Lay Language Generation via Semantic Invariance and Layperson-tailored Adaptationcited by 5
  20. 2025Activation Steering Meets Preference Optimization: Defense Against Jailbreaks in Vision Language Modelscited by 5
  21. 2026EmoShift: Lightweight Activation Steering for Enhanced Emotion-Aware Speech Synthesiscited by 5
  22. 2025Do LLMs Strategically Reveal, Conceal, and Infer Information? A Theoretical and Empirical Analysis in The Chameleon Gamecited by 4
  23. 2025A Mixture of Linear Corrections Generates Secure Codecited by 4
  24. 2025Agentic AI for autonomous anomaly management in complex systemscited by 4
  25. 2025Prototype-Based Dynamic Steering for Large Language Modelscited by 4
  26. 2026The Confidence Manifold: Geometric Structure of Correctness Representations in Language Modelscited by 4
  27. 2026Enhancing Instruction Following of LLMs via Activation Steering with Dynamic Rejectioncited by 4
  28. 2026Global Evolutionary Steering: Refining Activation Steering Control via Cross-Layer Consistencycited by 4
  29. 2026Steering at the Source: Style Modulation Heads for Robust Persona Controlcited by 4
  30. 2025But what is your honest answer? Aiding LLM-judges with honest alternatives using steering vectorscited by 3
  31. 2026Concept Heterogeneity-aware Representation Steeringcited by 3
  32. 2026PARALLAX: Separating Genuine Hallucination Detection from Benchmark Construction Artifactscited by 3
  33. 2025From Hallucinations to Jailbreaks: Rethinking the Vulnerability of Large Foundation Modelscited by 2
  34. 2025The Unintended Trade-off of AI Alignment:Balancing Hallucination Mitigation and Safety in LLMscited by 2
  35. 2025Activations as Features: Probing LLMs for Generalizable Essay Scoring Representationscited by 2
  36. 2026Hallucination Basins: A Dynamic Framework for Understanding and Controlling LLM Hallucinationscited by 2
  37. 2026Sparse Autoencoders for Interpretable Emotion Control in Text-to-Speechcited by 2
  38. 2026Deployable Per-Instance Multi-Layer Activation Steering for Large Language Modelscited by 2

50 most cited of 74 All 74 (JSON)