mlsanityData to 7 Oct 2026

Truth Forest: Toward Multi-Scale Truthfulness in Large Language Models through Intervention without Tuning

arXiv 2312.17484 ↗First posted 29 Dec 2023cs.CL cs.AIcited by 45

Not on the map. Why?

Abstract (arXiv)

Despite the great success of large language models (LLMs) in various tasks, they suffer from generating hallucinations. We introduce Truth Forest, a method that enhances truthfulness in LLMs by uncovering hidden truth representations using multi-dimensional orthogonal probes. Specifically, it creates multiple orthogonal bases for modeling truth by incorporating orthogonal constraints into the probes. Moreover, we introduce Random Peek, a systematic technique considering an extended range of positions within the sequence, reducing the gap between discerning and generating truth features in LLMs. By employing this approach, we improved the truthfulness of Llama-2-7B from 40.8% to 74.5% on TruthfulQA. Likewise, significant improvements are observed in fine-tuned models. We conducted a thorough analysis of truth features using probes. Our visualization results show that orthogonal probes capture complementary truth-related features, forming well-defined clusters that reveal the inherent structure of the dataset.

Key facts. Cited by 45 papers in the map (data through 7 Oct 2026). Builds on 11 of its 16 references that are in the map. First posted on arXiv 29 Dec 2023.

Builds on

11 of 16 references.

References · 11

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  5. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  6. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  7. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
  8. 2022Self-critiquing models for assisting human evaluatorscited by 372
  9. 2021More Control for Free! Image Synthesis with Semantic Diffusion Guidancecited by 211
  10. 2017Learning to Generate Reviews and Discovering Sentimentcited by 147
  11. 2020Language Models are Open Knowledge Graphscited by 95

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 20; 2025: 15; 2026: 10.

45 papers.

Sort:

By area:

  1. 2024Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Mapscited by 140
  2. 2024TruthX: Alleviating Hallucinations by Editing Large Language Models in Truthful Spacecited by 90
  3. 2024Semantics-Adaptive Activation Intervention for LLMs via Dynamic Steering Vectorscited by 61
  4. 2024Spectral Editing of Activations for Large Language Model Alignmentcited by 54
  5. 2024Internal Consistency and Self-Feedback in Large Language Models: A Surveycited by 49
  6. 2025Steer LLM Latents for Hallucination Detectioncited by 49
  7. 2024Knowledge Boundary of Large Language Models: A Surveycited by 36
  8. 2025Personalized Text Generation with Contrastive Activation Steeringcited by 30
  9. 2024HaluEval-Wild: Evaluating Hallucinations of Language Models in the Wildcited by 27
  10. 2024PrivacyRestore: Privacy-Preserving Inference in Large Language Models via Privacy Removal and Restorationcited by 27
  11. 2024Mitigating Large Language Model Hallucination with Faithful Finetuningcited by 24
  12. 2024On the Universal Truthfulness Hyperplane Inside LLMscited by 23
33 more
  1. 2024Constructing Benchmarks and Interventions for Combating Hallucinations in LLMscited by 19
  2. 2024Improving Factuality in Large Language Models via Decoding-Time Hallucinatory and Truthful Comparatorscited by 19
  3. 2024Is Factuality Enhancement a Free Lunch For LLMs? Better Factuality Can Lead to Worse Context-Faithfulnesscited by 17
  4. 2026Spherical Steering: Geometry-Aware Activation Rotation for Language Modelscited by 17
  5. 2024Distinguishing Ignorance from Error in LLM Hallucinationscited by 14
  6. 2024Non-Linear Inference Time Intervention: Improving LLM Truthfulnesscited by 9
  7. 2024NoVo: Norm Voting off Hallucinations with Attention Heads in Large Language Modelscited by 9
  8. 2025LF-Steering: Latent Feature Activation Steering for Enhancing Semantic Consistency in Large Language Modelscited by 7
  9. 2025SAFE: A Sparse Autoencoder-Based Framework for Robust Query Enrichment and Hallucination Mitigation in LLMscited by 7
  10. 2025Steering When Necessary: Flexible Steering Large Language Models with Backtrackingcited by 7
  11. 2025TraceDet: Hallucination Detection from the Decoding Trace of Diffusion Large Language Modelscited by 7
  12. 2024GRATH: Gradual Self-Truthifying for Large Language Modelscited by 6
  13. 2024Enhanced Language Model Truthfulness with Learnable Intervention and Uncertainty Expressioncited by 6
  14. 2025Loki's Dance of Illusions: A Comprehensive Survey of Hallucination in Large Language Modelscited by 6
  15. 2024Who Brings the Frisbee: Probing Hidden Hallucination Factors in Large Vision-Language Model via Causality Analysiscited by 4
  16. 2025ExpertSteer: Intervening in LLMs through Expert Knowledgecited by 4
  17. 2025Bridging the Knowledge-Prediction Gap in LLMs on Multiple-Choice Questionscited by 4
  18. 2025DSVD: Dynamic Self-Verify Decoding for Faithful Generation in Large Language Modelscited by 2
  19. 2025The Illusionist's Prompt: Exposing the Factual Vulnerabilities of Large Language Models with Linguistic Nuancescited by 2
  20. 2025Balancing Stylization and Truth via Disentangled Representation Steeringcited by 2
  21. 2026Exploring Diverse Generation Paths via Inference-time Stiefel Activation Steeringcited by 2
  22. 2025Expanding before Inferring: Enhancing Factuality in Large Language Models through Premature Layers Interpolationcited by 1
  23. 2024Lower Layers Matter: Alleviating Hallucination via Multi-Layer Fusion Contrastive Decoding with Truthfulness Refocusedcited by 0
  24. 2025One SPACE to Rule Them All: Jointly Mitigating Factuality and Faithfulness Hallucinations in LLMscited by 0
  25. 2025Look Closer! An Adversarial Parametric Editing Framework for Hallucination Mitigation in VLMscited by 0
  26. 2026Watch the Model Think: On-Policy Extraction of Activation Steering Vectorscited by 0
  27. 2026No One Size Fits All: QueryBandits for Hallucination Mitigationcited by 0
  28. 2026Three Concrete Challenges and Two Hopes for the Safety of Unsupervised Elicitationcited by 0
  29. 2026Hallucination-aware intermediate representation edit in large vision-language modelscited by 0
  30. 2026Micro-Macro Retrieval: Reducing Long-Form Hallucination in Large Language Modelscited by 0
  31. 2026CASS-RTL: Correctness-Aware Subspace Steering for RTL Generation with LLMscited by 0
  32. 2026Probabilistic Concept-Aware Steering for Trustworthy LLM Inferencecited by 0
  33. 2026Look Before You Leap: Factual Decoding with Internal Attribution Signalscited by 0