mlsanityData to 14 Oct 2026

TRACESTEER: Trace Driven Activation Steering for Hallucination Mitigation in Diffusion Language Models

arXiv 2610.18173 ↗First posted 13 Oct 2026cs.CLcited by 0

On the map: Diffusion models → Discrete and text diffusion

also in 1 more
Abstract (arXiv)

Diffusion large language models (dLLMs) have emerged as a promising alternative to autoregressive LLMs, yet remain susceptible to hallucination. Activation steering offers a promising mitigation approach by guiding parallel token predictions through internal representations. However, existing steering methods largely overlook two characteristics of dLLMs: dynamic denoising states and temporal hallucination risks, which challenge predefined steering vectors and uniform intervention strengths. We propose TRACESTEER, a trace-driven activation steering framework that uses preceding denoising predictions as feedback to jointly adapt steering direction and strength. Specifically, Denoising Feedback Steering (DF-Steering) constructs step-specific steering vectors from preceding predictions and current hidden states. Trajectory-Guided Steering Gate (TS-Gate) regulates their strength using prediction uncertainty and the denoising timestep. With the backbone frozen, factual supervision and magnitude regularization jointly guide the learning of adaptive steering interventions, promoting factual reliability and stability throughout denoising. Experiments on two dLLMs show improved factual reliability on TruthfulQA and generalization across five downstream benchmarks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 27 of its 40 references that are in the map. First posted on arXiv 13 Oct 2026.

Builds on

27 of 40 references.

Closest ancestors · 4

Same direction, newest first

  1. 2026TACG: Trajectory-Aware Commit Gating for Diffusion Language Model Decoding3 Jul
  2. 2026OSCAR: Orchestrated Self-verification and Cross-path Refinement2 Apr
  3. 2026Parallelism and Generation Order in Masked Diffusion Language Models: Limits Today, Potential Tomorrow22 Jan
  4. 2025Activation Steering for Masked Diffusion Language Models30 Dec

Other references · 23

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  4. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  5. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,183
  6. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  7. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  8. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  9. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  10. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 896
  11. 2016A Corpus and Evaluation Framework for Deeper Understanding of Commonsense Storiescited by 417
  12. 2023Trusting Your Evidence: Hallucinate Less with Context-aware Decodingcited by 327
11 more
  1. 2024TruthX: Alleviating Hallucinations by Editing Large Language Models in Truthful Spacecited by 91
  2. 2023Alleviating Hallucinations of Large Language Models through Induced Hallucinationscited by 79
  3. 2025The Hidden Life of Tokens: Reducing Hallucination of Large Vision-Language Models via Visual Information Steeringcited by 66
  4. 2025LinearRAG: Linear Graph Retrieval Augmented Generation on Large-scale Corporacited by 49
  5. 2026Spherical Steering: Geometry-Aware Activation Rotation for Language Modelscited by 21
  6. 2025Learning to Steer: Input-dependent Steering for Multimodal LLMscited by 15
  7. 2026DynHD: Hallucination Detection for Diffusion Large Language Models via Denoising Dynamics Deviation Learningcited by 15
  8. 2025TraceDet: Hallucination Detection from the Decoding Trace of Diffusion Large Language Modelscited by 8
  9. 2026ContextFocus: Activation Steering for Contextual Faithfulness in Large Language Modelscited by 4
  10. 2026TRE: Training-Free Hallucination Detection for Diffusion Language Modelscited by 1
  11. 2026DeMTS: Denoising Trajectories as Multivariate Time Series for Hallucination Detection in Diffusion Language Modelscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Lost in Diffusion: Uncovering Hallucination Patterns and Failure Modes in Diffusion Large Language Models
  2. 2026TDGNet: Hallucination Detection in Diffusion Language Models via Temporal Dynamic Graphs
  3. 2026Dynamic Multimodal Activation Steering for Hallucination Mitigation in Large Vision-Language Models
  4. 2026Mitigating Hallucination in Vision-Language Models through Barrier-Regulated Adaptive Closed-form Steering
  5. 2026Steer Where It Matters: Token-Level Visual-Sensitivity Steering for LVLMs Hallucination Mitigation
  6. 2025Hallucination reduction with CASAL: Contrastive Activation Steering For Amortized Learning
  7. 2025Mitigating Diffusion Model Hallucinations with Dynamic Guidance
  8. 2026Alleviating Hallucination in Reasoning Tasks with Training-Free Uncertainty-Guided Steering