Continuous Language Diffusion as a Decoder-Interface Problem
On the map: Diffusion models → Discrete and text diffusion
Abstract (arXiv)
Gaussian-corrupted sentence embeddings have no direct linguistic interpretation, yet continuous diffusion language models can generate fluent text from them. We study this puzzle through Embedded Language Flows (ELF) and identify a decoder-basin mechanism: our evidence suggests that denoising becomes reliable when trajectories reach regions where the native decoder can read stable tokens. We introduce a diagnostic protocol for denoisability, semantic recoverability, order sensitivity, decoder compatibility, and trajectory reliability. It exposes failures hidden by scalar metrics: low mean-squared error can discard linguistic content, low perplexity can reflect low-entropy collapse, and clean latent reconstruction can coexist with a narrow decoder basin. A decoder-margin bound explains why token recovery depends on margin and local decoder sensitivity, not latent error alone. Auditing public ELF checkpoints reveals an interface phase diagram: early predictions are weakly readable, mid-trajectory disagreement marks a competition region, and late predictions enter a high-margin decoder basin. Once inside, token realization is surprisingly simple on generated ELF states: frozen T5 (Text-to-Text Transfer Transformer) token-embedding lookup recovers 93--96% of native decoder decisions, and a single linear readout reaches 97.9% agreement at 32k samples, leaving an ≈1.1--1.2 perplexity gap in a structured residual tail. Under conservative held-out gates, a margin rule exits roughly 17--28% earlier in denoising steps under an explicit diagnostic monitor. Boundary checks on LangFlow, BitstreamDiffusion, and the Continuous Latent Diffusion Language Model (Cola-DLM) show that the same interface questions remain meaningful when the state object and decoder change. Continuous and latent diffusion language models should therefore be evaluated as representation-decoder systems.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. First posted on arXiv 7 Jun 2026.
Builds on
0 of 0 references.
No references here yet.
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Measuring Temporal Linguistic Emergence in Diffusion Language Models
- 2026Stability-Weighted Decoding for Diffusion Language Models
- 2026Window-Diffusion: Accelerating Diffusion Language Model Inference with Windowed Token Pruning and Caching
- 2025On the Role of Discreteness in Diffusion LLMs
- 2026ConvergeFlow: Language Flow with Provable Convergence to Token Embeddings
- 2026Flow Map Language Models: One-step Language Modeling via Continuous Denoising
- 2026CoDAR: Continuous Diffusion Language Models are More Powerful Than You Think
- 2026Locally Confident, Globally Stuck: The Quality-Exploration Dilemma in Diffusion Language Models