mlsanityData to 7 Oct 2026

Continuous Language Diffusion as a Decoder-Interface Problem

arXiv 2606.08810 ↗First posted 7 Jun 2026cs.CL cs.LGcited by 1

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Gaussian-corrupted sentence embeddings have no direct linguistic interpretation, yet continuous diffusion language models can generate fluent text from them. We study this puzzle through Embedded Language Flows (ELF) and identify a decoder-basin mechanism: our evidence suggests that denoising becomes reliable when trajectories reach regions where the native decoder can read stable tokens. We introduce a diagnostic protocol for denoisability, semantic recoverability, order sensitivity, decoder compatibility, and trajectory reliability. It exposes failures hidden by scalar metrics: low mean-squared error can discard linguistic content, low perplexity can reflect low-entropy collapse, and clean latent reconstruction can coexist with a narrow decoder basin. A decoder-margin bound explains why token recovery depends on margin and local decoder sensitivity, not latent error alone. Auditing public ELF checkpoints reveals an interface phase diagram: early predictions are weakly readable, mid-trajectory disagreement marks a competition region, and late predictions enter a high-margin decoder basin. Once inside, token realization is surprisingly simple on generated ELF states: frozen T5 (Text-to-Text Transfer Transformer) token-embedding lookup recovers 93--96% of native decoder decisions, and a single linear readout reaches 97.9% agreement at 32k samples, leaving an ≈1.1--1.2 perplexity gap in a structured residual tail. Under conservative held-out gates, a margin rule exits roughly 17--28% earlier in denoising steps under an explicit diagnostic monitor. Boundary checks on LangFlow, BitstreamDiffusion, and the Continuous Latent Diffusion Language Model (Cola-DLM) show that the same interface questions remain meaningful when the state object and decoder change. Continuous and latent diffusion language models should therefore be evaluated as representation-decoder systems.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. First posted on arXiv 7 Jun 2026.

Builds on

0 of 0 references.

No references here yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026Measuring Temporal Linguistic Emergence in Diffusion Language Models
  2. 2026Stability-Weighted Decoding for Diffusion Language Models
  3. 2026Window-Diffusion: Accelerating Diffusion Language Model Inference with Windowed Token Pruning and Caching
  4. 2025On the Role of Discreteness in Diffusion LLMs
  5. 2026ConvergeFlow: Language Flow with Provable Convergence to Token Embeddings
  6. 2026Flow Map Language Models: One-step Language Modeling via Continuous Denoising
  7. 2026CoDAR: Continuous Diffusion Language Models are More Powerful Than You Think
  8. 2026Locally Confident, Globally Stuck: The Quality-Exploration Dilemma in Diffusion Language Models