mlsanityData to 7 Oct 2026

Subliminal Clocks: Latent Time Modelling in Diffusion Language Models

arXiv 2607.01774 ↗First posted 2 Jul 2026cs.AI cs.CLcited by 0

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Diffusion Language Models (DLMs) have recently emerged as a promising alternative to autoregressive models. Unlike standard diffusion-based approaches, DLMs are not explicitly conditioned on a timestep, raising a natural question: do these models internally represent denoising progress, and how is such information used downstream? In this work, we show that DLMs do in fact encode a latent representation related to the diffusion timestep within their residual streams. We find that this signal can be reliably extracted using probes across layers, indicating that denoising progress is decodable from internal activations. We further demonstrate that steering the model along a low-dimensional subspace associated with the inferred timestep allows us to systematically modulate its notion of denoising progress, leading to predictable changes in model confidence and entropy. Finally, we analyse the geometry of the identified representation, showing that it exhibits structured and interpretable properties in activation space, and shedding light on how such a signal is processed by these models.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 31 of its 40 references that are in the map. First posted on arXiv 2 Jul 2026.

Builds on

31 of 40 references.

Closest ancestors · 14

Same direction, newest first

  1. 2026Steering Without Breaking: Mechanistically Informed Interventions for Discrete Diffusion Language Models8 May
  2. 2026Introspective Diffusion Language Models13 Apr
  3. 2026Diffusion Language Models Are Natively Length-Aware6 Mar
  4. 2026DLM-Scope: Mechanistic Interpretability of Diffusion Language Models via Sparse Autoencoders5 Feb
  5. 2025Masks Can Be Distracting: On Context Comprehension in Diffusion Language Models26 Nov
  6. 2025Attention Sinks in Diffusion Language Models17 Oct
  7. 2025SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generation7 Oct
  8. 2025Dream 7B: Diffusion Large Language Models21 Aug
  9. 2025Empirical Analysis of Decoding Biases in Masked Diffusion Models18 Aug
  10. 2025Large Language Diffusion Models14 Feb
  11. 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling4 Sep
  12. 2024Simple and Effective Masked Diffusion Language Models11 Jun
2 more
  1. 2024Simplified and Generalized Masked Diffusion for Discrete Data6 Jun
  2. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data6 Jun

Other references · 17

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2017Decoupled Weight Decay Regularizationcited by 1,722
  4. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
  5. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
  6. 2025Fast-dLLM v2: Efficient Block-Diffusion LLMcited by 128
  7. 2024Transformers represent belief state geometry in their residual streamcited by 73
  8. 2024Semantics-Adaptive Activation Intervention for LLMs via Dynamic Steering Vectorscited by 61
  9. 2025The Origins of Representation Manifolds in Large Language Modelscited by 46
  10. 2025The Devil behind the mask: An emergent safety vulnerability of Diffusion LLMscited by 32
  11. 2026Do Sparse Autoencoders Capture Concept Manifolds?cited by 25
  12. 2026Symmetry in language statistics shapes the geometry of model representationscited by 20
5 more
  1. 2025Large Language Models Encode Semantics and Alignment in Linearly Separable Representationscited by 14
  2. 2025Geometry of Decision Making in Language Modelscited by 8
  3. 2025Hypothesis-Driven Feature Manifold Analysis in LLMs via Supervised Multi-Dimensional Scalingcited by 6
  4. 2026R2-dLLM: Accelerating Diffusion Large Language Models via Spatio-Temporal Redundancy Reductioncited by 6
  5. 2026Generalized Discrete Diffusion from Snapshotscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Path-Dependent Denoising: A Non-Conservative Field Perspective on Order Collapse in Diffusion Language Models
  2. 2026On the Redundancy of Timestep Embeddings in Diffusion Models
  3. 2025Diffusion Language Models are Super Data Learners
  4. 2025A Survey on Diffusion Language Models
  5. 2025Time Is a Feature: Exploiting Temporal Dynamics in Diffusion Language Models
  6. 2026Not All Denoising Steps Are Equal: Model Scheduling for Faster Masked Diffusion Language Models
  7. 2025Inference-Time Scaling of Diffusion Language Models via Trajectory Refinement
  8. 2026TimeROME-DLM: Temporal Causal Tracing and Low-Rank Inference-Time Knowledge Editing for Masked Diffusion Language Models