# DiffuSeq-v2: Bridging Discrete and Continuous Text Spaces for Accelerated Seq2Seq Diffusion Models

arXiv: https://arxiv.org/abs/2310.05793 · HTML: https://mlsanity.com/paper/2310.05793/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Placed in Discrete and text diffusion, one of 14 directions in Diffusion models.
- Builds on 25 of its 44 references that are in the map.
- First posted on arXiv 9 Oct 2023.

## Placed in

- Diffusion models / [Discrete and text diffusion](https://mlsanity.com/diffusion/discrete-and-text-diffusion/)

## Builds on (most cited first, 25 of 25)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps](https://mlsanity.com/paper/2206.00927/) (2022)
- [Structured Denoising Diffusion Models in Discrete State-Spaces](https://mlsanity.com/paper/2107.03006/) (2021)
- [Diffusion-LM Improves Controllable Text Generation](https://mlsanity.com/paper/2205.14217/) (2022)
- [DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models](https://mlsanity.com/paper/2211.01095/) (2022)
- [Argmax Flows and Multinomial Diffusion: Learning Categorical Distributions](https://mlsanity.com/paper/2102.05379/) (2021)
- [Texygen: A Benchmarking Platform for Text Generation Models](https://mlsanity.com/paper/1802.01886/) (2018)
- [DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models](https://mlsanity.com/paper/2210.08933/) (2022)
- [Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioning](https://mlsanity.com/paper/2208.04202/) (2022)
- [Efficient Diffusion Training via Min-SNR Weighting Strategy](https://mlsanity.com/paper/2303.09556/) (2023)
- [DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models](https://mlsanity.com/paper/2211.15029/) (2022)
- [Scaling Neural Machine Translation](https://mlsanity.com/paper/1806.00187/) (2018)
- [A Reparameterized Discrete Diffusion Model for Text Generation](https://mlsanity.com/paper/2302.05737/) (2023)
- [Text Generation with Diffusion Language Models: A Pre-training Approach with Continuous Paragraph Denoise](https://mlsanity.com/paper/2212.11685/) (2022)
- [TESS: Text-to-Text Self-Conditioned Simplex Diffusion](https://mlsanity.com/paper/2305.08379/) (2023)
- [SeqDiffuSeq: Text Diffusion with Encoder-Decoder Transformers](https://mlsanity.com/paper/2212.10325/) (2022)
- [Unified Discrete Diffusion for Simultaneous Vision-Language Generation](https://mlsanity.com/paper/2211.14842/) (2022)
- [DiffuSum: Generation Enhanced Extractive Summarization with Diffusion](https://mlsanity.com/paper/2305.01735/) (2023)
- [A Cheaper and Better Diffusion Language Model with Soft-Masked Noise](https://mlsanity.com/paper/2304.04746/) (2023)
- [Diffusion-NAT: Self-Prompting Discrete Diffusion for Non-Autoregressive Text Generation](https://mlsanity.com/paper/2305.04044/) (2023)
- [Can Diffusion Model Achieve Better Performance in Text Generation? Bridging the Gap between Training and Inference!](https://mlsanity.com/paper/2305.04465/) (2023)

## Built on by (most cited first, 30 of 36)

- [Dream 7B: Diffusion Large Language Models](https://mlsanity.com/paper/2508.15487/) (2025)
- [Scaling Diffusion Language Models via Adaptation from Autoregressive Models](https://mlsanity.com/paper/2410.17891/) (2024)
- [DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generation](https://mlsanity.com/paper/2506.20639/) (2025)
- [Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planning](https://mlsanity.com/paper/2410.14157/) (2024)
- [How Discrete and Continuous Diffusion Meet: Comprehensive Analysis of Discrete Diffusion Models via a Stochastic Integral Framework](https://mlsanity.com/paper/2410.03601/) (2024)
- [Fast Solvers for Discrete Diffusion Models: Theory and Applications of High-Order Algorithms](https://mlsanity.com/paper/2502.00234/) (2025)
- [ELF: Embedded Language Flows](https://mlsanity.com/paper/2605.10938/) (2026)
- [Coevolutionary Continuous Discrete Diffusion: Make Your Diffusion Language Model a Latent Reasoner](https://mlsanity.com/paper/2510.03206/) (2025)
- [Diff-IP2D: Diffusion-Based Hand-Object Interaction Prediction on Egocentric Videos](https://mlsanity.com/paper/2405.04370/) (2024)
- [MADiff: Motion-Aware Mamba Diffusion Models for Hand Trajectory Prediction on Egocentric Videos](https://mlsanity.com/paper/2409.02638/) (2024)
- [Soft-Masked Diffusion Language Models](https://mlsanity.com/paper/2510.17206/) (2025)
- [Private Synthetic Text Generation with Diffusion Models](https://mlsanity.com/paper/2410.22971/) (2024)
- [DiffusionAttacker: Diffusion-Driven Prompt Manipulation for LLM Jailbreak](https://mlsanity.com/paper/2412.17522/) (2024)
- [Smoothie: Smoothing Diffusion on Token Embeddings for Text Generation](https://mlsanity.com/paper/2505.18853/) (2025)
- [IDLM: Inverse-distilled Diffusion Language Models](https://mlsanity.com/paper/2602.19066/) (2026)
- [Conditional [MASK] Discrete Diffusion Language Model](https://mlsanity.com/paper/2411.06438/) (2024)
- [DiffPO: Diffusion-styled Preference Optimization for Efficient Inference-Time Alignment of Large Language Models](https://mlsanity.com/paper/2503.04240/) (2025)
- [Segment-Level Diffusion: A Framework for Controllable Long-Form Generation with Diffusion Language Models](https://mlsanity.com/paper/2412.11333/) (2024)
- [Unifying Continuous and Discrete Text Diffusion with Non-simultaneous Diffusion Processes](https://mlsanity.com/paper/2505.22165/) (2025)
- [DiffuGR: Generative Document Retrieval with Diffusion Language Models](https://mlsanity.com/paper/2511.08150/) (2025)
- [Di2Pose: Discrete Diffusion Model for Occluded 3D Human Pose Estimation](https://mlsanity.com/paper/2405.17016/) (2024)
- [Meta-DiffuB: A Contextualized Sequence-to-Sequence Text Diffusion Model with Meta-Exploration](https://mlsanity.com/paper/2410.13201/) (2024)
- [DiffCap: Diffusion-based Real-time Human Motion Capture using Sparse IMUs and a Monocular Camera](https://mlsanity.com/paper/2508.06139/) (2025)
- [Negative-Aware Diffusion Process for Temporal Knowledge Graph Extrapolation](https://mlsanity.com/paper/2602.08815/) (2026)
- [FastDiSS: Few-step Match Many-step Diffusion Language Model on Sequence-to-Sequence Generation--Full Version](https://mlsanity.com/paper/2604.05551/) (2026)
- [Coupling Models for One-Step Discrete Generation](https://mlsanity.com/paper/2605.07193/) (2026)
- [Where Should Diffusion Enter a Language Model? Geometry-Guided Hidden-State Replacement](https://mlsanity.com/paper/2605.14368/) (2026)
- [IPED: An Implicit Perspective for Relational Triple Extraction based on Diffusion Model](https://mlsanity.com/paper/2403.00808/) (2024)
- [MoE-DiffuSeq: Enhancing Long-Document Diffusion Models with Sparse Attention and Mixture of Experts](https://mlsanity.com/paper/2512.20604/) (2025)
- [Table-to-Text Generation with Pretrained Diffusion Models](https://mlsanity.com/paper/2409.13739/) (2024)

## Abstract (arXiv)

Diffusion models have gained prominence in generating high-quality sequences of text. Nevertheless, current approaches predominantly represent discrete text within a continuous diffusion space, which incurs substantial computational overhead during training and results in slower sampling speeds. In this paper, we introduce a soft absorbing state that facilitates the diffusion model in learning to reconstruct discrete mutations based on the underlying Gaussian space, thereby enhancing its capacity to recover conditional signals. During the sampling phase, we employ state-of-the-art ODE solvers within the continuous space to expedite the sampling process. Comprehensive experimental evaluations reveal that our proposed method effectively accelerates the training convergence by 4x and generates samples of similar quality 800x faster, rendering it significantly closer to practical application. footnoteThe code is released at urlhttps://github.com/Shark-NLP/DiffuSeq