mlsanityData to 14 Oct 2026

Gumbel Straight Flow: Distilling Autoregressive Models into One-step Flow Maps

arXiv 2610.00497 ↗First posted 30 Sep 2026cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

We present Gumbel Straight Flow (GSF), a continuous flow map language model that leverages the noise-data coupling of a pretrained autoregressive language (AR) model. We theoretically demonstrate that the coupling between Gumbel noise and one-hot token sequences induced by an autoregressive model yields non-intersecting linear paths connecting the noise to the sequence representations. To further enhance high-quality few-step path sampling, we use a flow map semigroup objective where the tangent (velocity) condition is guided directly by the AR teacher. Across various benchmarks, including pretraining and downstream tasks, GSF can outperform current few-step language generation baselines.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 46 of its 56 references that are in the map. First posted on arXiv 30 Sep 2026.

Builds on

46 of 56 references.

References · 46

  1. 2022Scalable Diffusion Models with Transformerscited by 6,054
  2. 2022Flow Matching for Generative Modelingcited by 5,702
  3. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  4. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
  5. 2023Consistency Modelscited by 1,822
  6. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  7. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,317
  8. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,027
  9. 2023One-step Diffusion with Distribution Matching Distillationcited by 1,019
  10. 2023Improving and generalizing flow-based generative models with minibatch optimal transportcited by 879
  11. 2024Simple and Effective Masked Diffusion Language Modelscited by 848
  12. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 717
34 more
  1. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 626
  2. 2025Mean Flows for One-step Generative Modelingcited by 561
  3. 2023Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusioncited by 459
  4. 2023TinyStories: How Small Can Language Models Be and Still Speak Coherent English?cited by 449
  5. 2023InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generationcited by 395
  6. 2013One Billion Word Benchmark for Measuring Progress in Statistical Language Modelingcited by 308
  7. 2023Multisample Flow Matching: Straightening Flows with Minibatch Couplingscited by 278
  8. 2022Continuous diffusion for categorical datacited by 214
  9. 2025The Diffusion Dualitycited by 135
  10. 2025Accelerating Diffusion LLMs via Adaptive Parallel Decodingcited by 112
  11. 2025How to build a consistency model: Learning flow maps via self-distillationcited by 103
  12. 2024Variational Flow Matching for Graph Generationcited by 83
  13. 2024Multistep Distillation of Diffusion Models via Moment Matchingcited by 82
  14. 2024Beyond Autoregression: Fast LLMs via Self-Distillation Through Timecited by 78
  15. 2024CLLMs: Consistency Large Language Modelscited by 72
  16. 2026Flow Map Language Models: One-step Language Modeling via Continuous Denoisingcited by 66
  17. 2024Distillation of Discrete Diffusion through Dimensional Correlationscited by 48
  18. 2026ELF: Embedded Language Flowscited by 45
  19. 2026Categorical Flow Mapscited by 44
  20. 2025CANDI: Hybrid Discrete-Continuous Diffusion Modelscited by 34
  21. 2026Discrete Flow Mapscited by 29
  22. 2025Encoder-Decoder Diffusion Language Models for Efficient Training and Inferencecited by 24
  23. 2026Language Modeling with Hyperspherical Flowscited by 22
  24. 2016Causal transport in discrete time and applicationscited by 21
  25. 2025Ultra-Fast Language Generation via Discrete Diffusion Divergence Instructcited by 16
  26. 2026Self-conditioned Flow Map Language Models via Fixed-point Flowscited by 14
  27. 2025Fast and Accurate Causal Parallel Decoding using Jacobi Forcingcited by 13
  28. 2026Beyond Single Tokens: Distilling Discrete Diffusion Models via Discrete MMDcited by 12
  29. 2026Posterior Refinement: Fast Language Generation via Any-Order Flow Mapscited by 11
  30. 2026IDLM: Inverse-distilled Diffusion Language Modelscited by 8
  31. 2025AlignFlow: Improving Flow-based Generative Models with Semi-Discrete Optimal Transportcited by 7
  32. 2025Parallel Token Prediction for Language Modelscited by 7
  33. 2026Hacking Generative Perplexity: Why Unconditional Text Evaluation Needs Distributional Metricscited by 7
  34. 2026Gumbel Distillation for Parallel Text Generationcited by 1

Built on by

Not cited yet.