mlsanityData to 7 Oct 2026

DiffuSeq-v2: Bridging Discrete and Continuous Text Spaces for Accelerated Seq2Seq Diffusion Models

arXiv 2310.05793 ↗First posted 9 Oct 2023cs.LG cs.CLcited by 36

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Diffusion models have gained prominence in generating high-quality sequences of text. Nevertheless, current approaches predominantly represent discrete text within a continuous diffusion space, which incurs substantial computational overhead during training and results in slower sampling speeds. In this paper, we introduce a soft absorbing state that facilitates the diffusion model in learning to reconstruct discrete mutations based on the underlying Gaussian space, thereby enhancing its capacity to recover conditional signals. During the sampling phase, we employ state-of-the-art ODE solvers within the continuous space to expedite the sampling process. Comprehensive experimental evaluations reveal that our proposed method effectively accelerates the training convergence by 4x and generates samples of similar quality 800x faster, rendering it significantly closer to practical application. footnoteThe code is released at urlhttps://github.com/Shark-NLP/DiffuSeq

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 25 of its 44 references that are in the map. First posted on arXiv 9 Oct 2023.

Builds on

25 of 44 references.

Closest ancestors · 9

Same direction, newest first

  1. 2023TESS: Text-to-Text Self-Conditioned Simplex Diffusion15 May
  2. 2023Can Diffusion Model Achieve Better Performance in Text Generation? Bridging the Gap between Training and Inference!8 May
  3. 2023Diffusion-NAT: Self-Prompting Discrete Diffusion for Non-Autoregressive Text Generation6 May
  4. 2023A Cheaper and Better Diffusion Language Model with Soft-Masked Noise10 Apr
  5. 2023A Reparameterized Discrete Diffusion Model for Text Generation11 Feb
  6. 2022Text Generation with Diffusion Language Models: A Pre-training Approach with Continuous Paragraph Denoise22 Dec
  7. 2022SeqDiffuSeq: Text Diffusion with Encoder-Decoder Transformers20 Dec
  8. 2022DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models28 Nov
  9. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models17 Oct

Other references · 16

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2020Denoising Diffusion Implicit Modelscited by 9,655
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  4. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  5. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  6. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,961
  7. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  8. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  9. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Modelscited by 924
  10. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 566
  11. 2018Texygen: A Benchmarking Platform for Text Generation Modelscited by 484
  12. 2022Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioningcited by 362
4 more
  1. 2023Efficient Diffusion Training via Min-SNR Weighting Strategycited by 242
  2. 2018Scaling Neural Machine Translationcited by 204
  3. 2022Unified Discrete Diffusion for Simultaneous Vision-Language Generationcited by 39
  4. 2023DiffuSum: Generation Enhanced Extractive Summarization with Diffusioncited by 25

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 14; 2025: 13; 2026: 9.

36 papers.

Sort:

By area:

  1. 2025Dream 7B: Diffusion Large Language Modelscited by 515
  2. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Modelscited by 268
  3. 2025DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generationcited by 184
  4. 2024Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planningcited by 135
  5. 2024How Discrete and Continuous Diffusion Meet: Comprehensive Analysis of Discrete Diffusion Models via a Stochastic Integral Frameworkcited by 48
  6. 2025Fast Solvers for Discrete Diffusion Models: Theory and Applications of High-Order Algorithmscited by 47
  7. 2025Coevolutionary Continuous Discrete Diffusion: Make Your Diffusion Language Model a Latent Reasonercited by 28
  8. 2026ELF: Embedded Language Flowscited by 27
  9. 2024Diff-IP2D: Diffusion-Based Hand-Object Interaction Prediction on Egocentric Videoscited by 21
  10. 2024MADiff: Motion-Aware Mamba Diffusion Models for Hand Trajectory Prediction on Egocentric Videoscited by 17
  11. 2025Soft-Masked Diffusion Language Modelscited by 15
  12. 2024Private Synthetic Text Generation with Diffusion Modelscited by 10
24 more
  1. 2024DiffusionAttacker: Diffusion-Driven Prompt Manipulation for LLM Jailbreakcited by 8
  2. 2025Smoothie: Smoothing Diffusion on Token Embeddings for Text Generationcited by 8
  3. 2025DiffPO: Diffusion-styled Preference Optimization for Efficient Inference-Time Alignment of Large Language Modelscited by 5
  4. 2026IDLM: Inverse-distilled Diffusion Language Modelscited by 5
  5. 2024Conditional [MASK] Discrete Diffusion Language Modelcited by 4
  6. 2024Segment-Level Diffusion: A Framework for Controllable Long-Form Generation with Diffusion Language Modelscited by 4
  7. 2025Unifying Continuous and Discrete Text Diffusion with Non-simultaneous Diffusion Processescited by 4
  8. 2024Di2Pose: Discrete Diffusion Model for Occluded 3D Human Pose Estimationcited by 2
  9. 2024Meta-DiffuB: A Contextualized Sequence-to-Sequence Text Diffusion Model with Meta-Explorationcited by 2
  10. 2025DiffCap: Diffusion-based Real-time Human Motion Capture using Sparse IMUs and a Monocular Cameracited by 2
  11. 2025DiffuGR: Generative Document Retrieval with Diffusion Language Modelscited by 2
  12. 2026Negative-Aware Diffusion Process for Temporal Knowledge Graph Extrapolationcited by 2
  13. 2026FastDiSS: Few-step Match Many-step Diffusion Language Model on Sequence-to-Sequence Generation--Full Versioncited by 2
  14. 2024IPED: An Implicit Perspective for Relational Triple Extraction based on Diffusion Modelcited by 1
  15. 2025MoE-DiffuSeq: Enhancing Long-Document Diffusion Models with Sparse Attention and Mixture of Expertscited by 1
  16. 2026Coupling Models for One-Step Discrete Generationcited by 1
  17. 2026Where Should Diffusion Enter a Language Model? Geometry-Guided Hidden-State Replacementcited by 1
  18. 2024Table-to-Text Generation with Pretrained Diffusion Modelscited by 0
  19. 2024Discrete Modeling via Boundary Conditional Diffusion Processescited by 0
  20. 2025DiffER: Categorical Diffusion for Chemical Retrosynthesiscited by 0
  21. 2025SA-DiffuSeq: Addressing Computational and Scalability Challenges in Long-Document Generation with Sparse Attentioncited by 0
  22. 2026Language Generation as Optimal Control: Closed-Loop Diffusion in Latent Control Spacecited by 0
  23. 2026FlowLM: Few-Step Language Modeling via Diffusion-to-Flow Adaptationcited by 0
  24. 2026When to Plan, When to Polish: Noise Level as a Granularity Axis for Diffusion Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025The Diffusion Duality
  2. 2025Your Absorbing Discrete Diffusion Secretly Models the Bayesian Posterior
  3. 2025Encoder-Decoder Diffusion Language Models for Efficient Training and Inference
  4. 2025Test-Time Anchoring for Discrete Diffusion Posterior Sampling
  5. 2022DiffusER: Discrete Diffusion via Edit-based Reconstruction
  6. 2025Accelerated Diffusion Models via Speculative Sampling
  7. 2025FlashDLM: Accelerating Diffusion Language Model Inference via Efficient KV Caching and Guided Diffusion
  8. 2025Diffusion Language Model Inference with Monte Carlo Tree Search