mlsanityData to 14 Oct 2026

ILRR: Inference-Time Steering Method for Masked Diffusion Language Models

arXiv 2601.21647 ↗First posted 29 Jan 2026cs.CL cs.AI cs.LGcited by 4

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Discrete Diffusion Language Models (DLMs) offer a promising non-autoregressive alternative for text generation, yet effective mechanisms for inference-time control remain relatively underexplored. Existing approaches include sampling-level guidance or trajectory optimization mechanisms. In this work, we study the paradigm of reference-based latent steering for DLMs. We introduce Iterative Latent Representation Refinement (ILRR), an efficient framework for steering DLMs using a reference text as a high-level semantic blueprint. ILRR extracts and injects reference-derived semantic signals into the evolving activations of the generated sequence, enabling tunable transfer of coarse properties such as sentiment. We further introduce Spatially Modulated Steering, an extension that enables long-form generation to be guided by shorter references by regulating intensity across the sequence. Empirically, we demonstrate that ILRR achieves effective control on LLaDA and MDLM architectures with low computational overhead, requiring only one additional parallel forward pass per denoising step. Under comparable compute budgets, ILRR improves attribute accuracy over baselines by 10% to 60% points. Our results suggest that the iterative, global denoising process makes DLMs a natural substrate for effective sequence-wide activation-level control.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 44 of its 61 references that are in the map. First posted on arXiv 29 Jan 2026.

Builds on

44 of 61 references.

Closest ancestors · 15

Same direction, newest first

  1. 2025WeDLM: Reconciling Diffusion Language Models with Standard Causal Attention for Fast Inference28 Dec
  2. 2025Learning Unmasking Policies for Diffusion Language Models9 Dec
  3. 2025Dream 7B: Diffusion Large Language Models21 Aug
  4. 2025Improving Text Style Transfer using Masked Diffusion Language Models with Inference-time Scaling14 Aug
  5. 2025Discrete Diffusion in Large Language and Multimodal Models: A Survey16 Jun
  6. 2025Generalized Interpolating Discrete Diffusion6 Mar
  7. 2025Remasking Discrete Diffusion Models with Inference-Time Scaling1 Mar
  8. 2025Implicit Search via Discrete Diffusion: A Study on Chess27 Feb
  9. 2025Path Planning for Masked Diffusion Model Sampling5 Feb
  10. 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling4 Sep
  11. 2024Simple and Effective Masked Diffusion Language Models11 Jun
  12. 2024Simplified and Generalized Masked Diffusion for Discrete Data6 Jun
3 more
  1. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct
  2. 2022DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models28 Nov
  3. 2022SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Control31 Oct

Other references · 29

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  4. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  5. 2018Deep contextualized word representationscited by 2,290
  6. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  7. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  8. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
  9. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  10. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,059
  11. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
  12. 2023Steering Language Models With Activation Engineeringcited by 922
17 more
  1. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 906
  2. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 799
  3. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 716
  4. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Modelscited by 635
  5. 2021DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Expertscited by 441
  6. 2020TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classificationcited by 417
  7. 2020GeDi: Generative Discriminator Guided Sequence Generationcited by 378
  8. 2021FUDGE: Controlled Text Generation With Future Discriminatorscited by 376
  9. 2023In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steeringcited by 245
  10. 2025A General Framework for Inference-time Scaling and Steering of Diffusion Modelscited by 224
  11. 2024Programming Refusal with Conditional Activation Steeringcited by 206
  12. 2024Improving Instruction-Following in Language Models through Activation Steeringcited by 160
  13. 2024Simple Guidance Mechanisms for Discrete Diffusion Modelscited by 155
  14. 2024Derivative-Free Guidance in Continuous and Discrete Diffusion Models with Soft Value-Based Decodingcited by 136
  15. 2024Semantics-Adaptive Activation Intervention for LLMs via Dynamic Steering Vectorscited by 63
  16. 2025Inference-time Scaling of Diffusion Models through Classical Searchcited by 49
  17. 2025Beyond Prompt Engineering: Robust Behavior Control in LLMs via Steering Target Atomscited by 24

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Inference-Time Alignment of Diffusion Models via Trust-Region Iterative Twisted Sequential Monte Carlocited by 6
  2. 2026Steering Without Breaking: Mechanistically Informed Interventions for Discrete Diffusion Language Modelscited by 3
  3. 2026Focus on the Core: Empowering Diffusion Large Language Models by Self-Contrastcited by 0
  4. 2026Noise Out, Bias In: Targeted Bias Injection in Diffusion Language Models via Closed-Loop Activation Steeringcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026DLM-SWAI: Steering Diffusion Language Models Before They Unmask
  2. 2025Activation Steering for Masked Diffusion Language Models
  3. 2026DiLaDiff: Distilled Latent-Augmented Diffusion for Language Modeling
  4. 2026TimpaTeks: Automatic In-place Text Sequence Modification via Diffusion Language Model Steering
  5. 2026Diffusion Language Models: An Experimental Analysis
  6. 2025Latent-Augmented Discrete Diffusion Models
  7. 2025A Survey on Diffusion Language Models
  8. 2026Continuous Latent Diffusion Language Model