# Improving Few-Step Language Flows with Untied Self-Conditioning

arXiv: https://arxiv.org/abs/2608.22244 · HTML: https://mlsanity.com/paper/2608.22244/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Discrete and text diffusion, one of 14 directions in Diffusion models.
- Builds on 26 of its 30 references that are in the map.
- First posted on arXiv 23 Aug 2026.

## Placed in

- Diffusion models / [Discrete and text diffusion](https://mlsanity.com/diffusion/discrete-and-text-diffusion/)

## Builds on (most cited first, 26 of 26)

- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Diffusion-LM Improves Controllable Text Generation](https://mlsanity.com/paper/2205.14217/) (2022)
- [DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models](https://mlsanity.com/paper/2211.01095/) (2022)
- [Fast Sampling of Diffusion Models with Exponential Integrator](https://mlsanity.com/paper/2204.13902/) (2022)
- [From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipeline](https://mlsanity.com/paper/2406.11939/) (2024)
- [DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models](https://mlsanity.com/paper/2210.08933/) (2022)
- [UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Models](https://mlsanity.com/paper/2302.04867/) (2023)
- [Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioning](https://mlsanity.com/paper/2208.04202/) (2022)
- [SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Control](https://mlsanity.com/paper/2210.17432/) (2022)
- [Likelihood-Based Diffusion Language Models](https://mlsanity.com/paper/2305.18619/) (2023)
- [Self-conditioned Embedding Diffusion for Text Generation](https://mlsanity.com/paper/2211.04236/) (2022)
- [Step-unrolled Denoising Autoencoders for Text Generation](https://mlsanity.com/paper/2112.06749/) (2021)
- [TESS: Text-to-Text Self-Conditioned Simplex Diffusion](https://mlsanity.com/paper/2305.08379/) (2023)
- [Boosting Diffusion Models with Moving Average Sampling in Frequency Domain](https://mlsanity.com/paper/2403.17870/) (2024)
- [TESS 2: A Large-Scale Generalist Diffusion Language Model](https://mlsanity.com/paper/2502.13917/) (2025)
- [Self-Guidance: Boosting Flow and Diffusion Generation on Their Own](https://mlsanity.com/paper/2412.05827/) (2024)
- [TEncDM: Understanding the Properties of the Diffusion Model in the Space of Language Model Encodings](https://mlsanity.com/paper/2402.19097/) (2024)
- [Continuous Diffusion Scales Competitively with Discrete Diffusion for Language](https://mlsanity.com/paper/2605.18530/) (2026)
- [Self-conditioned Flow Map Language Models via Fixed-point Flows](https://mlsanity.com/paper/2607.00714/) (2026)
- [Simple Self-Conditioning Adaptation for Masked Diffusion Models](https://mlsanity.com/paper/2604.26985/) (2026)
- [Diffusion Sampling with Momentum for Mitigating Divergence Artifacts](https://mlsanity.com/paper/2307.11118/) (2023)
- [Text Diffusion with Reinforced Conditioning](https://mlsanity.com/paper/2402.14843/) (2024)
- [HiGS: History-Guided Sampling for Plug-and-Play Enhancement of Diffusion Models](https://mlsanity.com/paper/2509.22300/) (2025)
- [FastDiSS: Few-step Match Many-step Diffusion Language Model on Sequence-to-Sequence Generation--Full Version](https://mlsanity.com/paper/2604.05551/) (2026)
- [Low Perplexity is Repetition: A One-Dimensional Self-Conditioning Attractor in Continuous Diffusion LMs](https://mlsanity.com/paper/2607.00588/) (2026)
- [DiFA: Inference-Time Forward-Process Alignment for Diffusion Models](https://mlsanity.com/paper/2607.17972/) (2026)

## Abstract (arXiv)

Flow-matching language models refine all token positions in parallel and can trade sampling steps for latency, yet generation quality still degrades sharply with few sampling steps. We trace a source of this degradation to a train--inference mismatch in previous-prediction self-conditioning: during training, the self-conditioning input is computed from the current noisy state with no intervening solver step; during sampling, the solver folds the previous prediction into the latent before that same prediction reappears as the explicit self-conditioning input. This coupling, absent during training, creates redundancy that grows with step width. We show that the mismatch degrades both the self-conditioning input and the solver update, and derive a correction for each from the model's own structure. From the frozen projection weights we identify directions along which the self-conditioning input is redundant with the latent and dampen them; from the solver's integration structure we derive that a step-average prediction is needed and approximate it from prediction history, with scale set by offline trajectory statistics. The resulting sampler, Untied Self-Conditioning, requires no retraining and uses one evaluation per step. At 8 sampling steps on LangFlow, it reduces OpenWebText generative perplexity from 531 to~62 (8.6×); under an adapted Arena-Hard-Auto~v2 protocol, its outputs are preferred in 96% of pairwise comparisons. On ELF-B it reduces generative perplexity from 71 to~43. Improvements hold from 8 to 256 sampling steps.