mlsanityData to 7 Oct 2026

ConvergeFlow: Language Flow with Provable Convergence to Token Embeddings

arXiv 2608.23551 ↗First posted 24 Aug 2026cs.CL cs.AI cs.LGcited by 2

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Recent advances in continuous diffusion and flow-based language models (LMs) have achieved performance competitive with discrete LMs. However, existing continuous frameworks still rely on decoders supervised with cross entropy (CE) because the flow trajectories are not guaranteed to terminate at valid token embeddings. Motivated by this limitation, we introduce ConvergeFlow, an embedding-space flow-based LM, which constrains the data predictor to the convex hull of token embeddings and trains it solely with the mean squared error objective induced by flow matching. Under suitable regularity conditions, we prove that the resulting flow converges to valid token embeddings despite errors in the data predictor, enabling direct token prediction without a CE-supervised decoder. We further develop three sampling mechanisms for controlling the trade-off between the generative perplexity and entropy. Experiments on OpenWebText demonstrate that ConvergeFlow achieves performance competitive with existing continuous and discrete diffusion LMs. These findings demonstrate the potential of the flow-based paradigm for language modeling. Our code is available at https://github.com/Na-Li66/ConvergeFlow.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 83 of its 88 references that are in the map. First posted on arXiv 24 Aug 2026.

Builds on

83 of 88 references.

Closest ancestors · 34

Same direction, newest first

  1. 2026ELF: Embedded Language Flows11 May
  2. 2026How to Train Your Latent Diffusion Language Model Jointly With the Latent Space8 May
  3. 2026Continuous Latent Diffusion Language Model7 May
  4. 2026LangFlow: Continuous Diffusion Rivals Discrete in Language Modeling13 Apr
  5. 2026Confidence-Based Decoding is Provably Efficient for Diffusion Language Models23 Mar
  6. 2026Adaptation to Intrinsic Dependence in Diffusion Language Models23 Feb
  7. 2026Flow Map Language Models: One-step Language Modeling via Continuous Denoising18 Feb
  8. 2026Scaling Beyond Masked Diffusion Language Models16 Feb
  9. 2025From Bits to Rounds: Parallel Decoding with Exploration for Diffusion Language Models26 Nov
  10. 2025Optimal Inference Schedules for Masked Diffusion Models6 Nov
  11. 2025Error Bounds and Optimal Schedules for Masked Diffusions with Factorized Approximations29 Oct
  12. 2025Dream 7B: Diffusion Large Language Models21 Aug
22 more
  1. 2025Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inference4 Aug
  2. 2025Cosmos: Compressed and Smooth Latent Space for Text Diffusion Modeling26 Jun
  3. 2025The Diffusion Duality12 Jun
  4. 2025Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmasking30 May
  5. 2025Breaking AR's Sampling Bottleneck: Provable Acceleration via Diffusion Language Models27 May
  6. 2025Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decoding22 May
  7. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models12 Mar
  8. 2025TESS 2: A Large-Scale Generalist Diffusion Language Model19 Feb
  9. 2025Large Language Diffusion Models14 Feb
  10. 2025Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions10 Feb
  11. 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling4 Sep
  12. 2024Simple and Effective Masked Diffusion Language Models11 Jun
  13. 2024Simplified and Generalized Masked Diffusion for Discrete Data6 Jun
  14. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data6 Jun
  15. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct
  16. 2023PLANNER: Generating Diversified Paragraph via Latent Language Diffusion Model5 Jun
  17. 2023Likelihood-Based Diffusion Language Models30 May
  18. 2023TESS: Text-to-Text Self-Conditioned Simplex Diffusion15 May
  19. 2022Latent Diffusion for Language Generation19 Dec
  20. 2022Continuous diffusion for categorical data28 Nov
  21. 2022SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Control31 Oct
  22. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models17 Oct

Other references · 49

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  7. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  8. 2022Scalable Diffusion Models with Transformerscited by 5,931
  9. 2022Classifier-Free Diffusion Guidancecited by 5,717
  10. 2022Flow Matching for Generative Modelingcited by 5,426
  11. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  12. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
37 more
  1. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  2. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
  3. 2022Video Diffusion Modelscited by 2,280
  4. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,961
  5. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  6. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  7. 2023One-step Diffusion with Distribution Matching Distillationcited by 978
  8. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Modelscited by 924
  9. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 566
  10. 2022Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptionscited by 440
  11. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 437
  12. 2023Improved Techniques for Training Consistency Modelscited by 430
  13. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 404
  14. 2022Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioningcited by 362
  15. 2023Nearly d-Linear Convergence Bounds for Diffusion Models via Stochastic Localizationcited by 230
  16. 2022Diffusion probabilistic modeling of protein backbones in 3D for the motif-scaffolding problemcited by 229
  17. 2025Mercury: Ultra-Fast Language Models Based on Diffusioncited by 225
  18. 2022Improved Analysis of Score-based Generative Modeling: User-Friendly Bounds under Minimal Smoothness Assumptionscited by 215
  19. 2022Convergence for score-based generative modeling with polynomial complexitycited by 176
  20. 2023Diffusion Models are Minimax Optimal Distribution Estimatorscited by 159
  21. 2023The probability flow ODE is provably fastcited by 145
  22. 2025LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuningcited by 145
  23. 2022Concrete Score Matching: Generalized Score Matching for Discrete Datacited by 144
  24. 2024A Sharp Convergence Theory for The Probability Flow ODEs of Diffusion Modelscited by 64
  25. 2024O(d/T) Convergence Theory for Diffusion Probabilistic Models under Minimal Assumptionscited by 48
  26. 2026Categorical Flow Mapscited by 37
  27. 2024Provable Acceleration for Diffusion Models under Minimal Assumptionscited by 27
  28. 2025Minimax Optimality of the Probability Flow ODE for Diffusion Modelscited by 23
  29. 2025Dimension-Free Convergence of Diffusion Models for Approximate Gaussian Mixturescited by 23
  30. 2026Discrete Flow Mapscited by 23
  31. 2024Instance-dependent Convergence Theory for Diffusion Modelscited by 21
  32. 2026Efficient Sampling with Discrete Diffusion Models: Sharp and Adaptive Guaranteescited by 19
  33. 2025Faster Diffusion Models via Higher-Order Approximationcited by 16
  34. 2025Optimal Convergence Analysis of DDPM for General Distributionscited by 16
  35. 2025A Sharp KL-Convergence Analysis for Diffusion Models under Minimal Assumptionscited by 8
  36. 2026The data geometry of masking diffusion: Certified-optimal schedules via unmasking growth complexitycited by 8
  37. 2026Diffusion Models Are Statistically Optimal for Learning Low-Dimensional Multi-Modal Distributionscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Scaling and Distilling Text Embeddings for Better Diffusibilitycited by 0
  2. 2026Training Parallel Speculative Draft Models by Directly Minimizing Expected Decoding Roundscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Continuous Language Diffusion as a Decoder-Interface Problem
  2. 2026Sampling from Flow Language Models via Marginal-Conditioned Bridges
  3. 2026FlowLM: Few-Step Language Modeling via Diffusion-to-Flow Adaptation
  4. 2026Masked Language Flow Models
  5. 2026Denoising Hierarchical Representations: Joint Continuous Diffusion for Language Modeling
  6. 2025Finish First, Perfect Later: Test-Time Token-Level Cross-Validation for Diffusion Large Language Models
  7. 2026Alpha Diffusion Language Models: Factorization Alone Is Not the Problem
  8. 2024CaLMFlow: Volterra Flow Matching using Causal Language Models