mlsanityData to 14 Oct 2026

TRACE: Trajectory Correction from Cross-layer Evidence for Hallucination Reduction

arXiv 2605.18163 ↗First posted 18 May 2026cs.AI cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Hallucination correction is not a one-direction problem. We show that intermediate layers are neither uniformly more truthful than final layers nor uniformly less trustworthy. Yet hallucination reduction is usually instantiated through one fixed intervention form: contrast one layer against another, steer along a truthfulness direction, or defer to external evidence. This framing is structurally incomplete. Cross-layer factual evidence does not evolve uniformly: in some failures truthful support is present internally and later suppressed, whereas in others candidate competition remains genuinely multi-directional across depth, so no single signed scalar family is generally sufficient. We introduce Trajectory Correction from Cross-layer Evidence for Hallucination Reduction (TRACE), a deterministic, training-free algorithm which corrects hallucinations at inference time by deriving both the corrective layer and the appropriate correction operator from each input's cross-layer candidate trajectory inside the LLM's own forward pass. Under one frozen hyperparameter setting, TRACE selects among scalar reversal, earlier-state recovery, and candidate-space correction using only model-internal evidence. Evaluated as a single universal algorithm across 15 models, 8 model families, and 3 factuality benchmarks, TRACE improves every evaluation cell, yielding mean gains of +12.26 MC1 points and +8.65 MC2-style points with no regressions, with gains reaching +47.20 MC1 and +43.38 MC2-style points. The method uses no labels, retrieval, pretraining, finetuning, or per-model calibration.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 42 of its 50 references that are in the map. First posted on arXiv 18 May 2026.

Builds on

42 of 50 references.

References · 42

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  5. 2025Qwen3 Technical Reportcited by 8,820
  6. 2016Layer Normalizationcited by 5,669
  7. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  8. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  9. 2025Gemma 3 Technical Reportcited by 1,824
  10. 2024Mixtral of Expertscited by 1,823
  11. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  12. 2019Root Mean Square Layer Normalizationcited by 1,480
30 more
  1. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,386
  2. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  3. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  4. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  5. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
  6. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 767
  7. 2024Phi-4 Technical Reportcited by 711
  8. 2023The Internal State of an LLM Knows When It's Lyingcited by 659
  9. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 623
  10. 2023DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Modelscited by 376
  11. 2023HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Modelscited by 341
  12. 2023Chain-of-Verification Reduces Hallucination in Large Language Modelscited by 324
  13. 2024INSIDE: LLMs' Internal States Retain the Power of Hallucination Detectioncited by 323
  14. 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 287
  15. 2024LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinationscited by 244
  16. 2025Phi-4-reasoning Technical Reportcited by 151
  17. 2026Ministral 3cited by 114
  18. 2024Unsupervised Real-Time Hallucination Detection based on the Internal States of Large Language Modelscited by 86
  19. 2024LLM Internal States Reveal Hallucination Risk Faced With a Querycited by 82
  20. 2024Adaptive Activation Steering: A Tuning-Free LLM Truthfulness Improvement Method for Diverse Hallucinations Categoriescited by 77
  21. 2024Factuality of Large Language Models: A Surveycited by 66
  22. 2024On Large Language Models' Hallucination with Regard to Known Factscited by 43
  23. 2024Mechanistic Understanding and Mitigation of Language Model Non-Factual Hallucinationscited by 38
  24. 2024SLED: Self Logits Evolution Decoding for Improving Factuality in Large Language Modelscited by 16
  25. 2024SH2: Self-Highlighted Hesitation Helps You Decode More Truthfullycited by 15
  26. 2024PoLLMgraph: Unraveling Hallucinations in Large Language Models via State Transition Dynamicscited by 14
  27. 2025Active Layer-Contrastive Decoding Reduces Hallucination in Large Language Model Generationcited by 10
  28. 2025Improve Decoding Factuality by Token-wise Cross Layer Entropy of Large Language Modelscited by 7
  29. 2024Enhanced Language Model Truthfulness with Learnable Intervention and Uncertainty Expressioncited by 6
  30. 2025DeLTa: A Decoding Strategy based on Logit Trajectory Prediction Improves Factuality and Reasoning Abilitycited by 3

Built on by

Not cited yet.