Data to 14 Oct 2026

TESS: Text-to-Text Self-Conditioned Simplex Diffusion

arXiv 2305.08379 ↗First posted 15 May 2023cs.CL cs.LGcited by 65

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Diffusion models have emerged as a powerful paradigm for generation, obtaining strong performance in various continuous domains. However, applying continuous diffusion models to natural language remains challenging due to its discrete nature and the need for a large number of diffusion steps to generate text, making diffusion-based generation expensive. In this work, we propose Text-to-text Self-conditioned Simplex Diffusion (TESS), a text diffusion model that is fully non-autoregressive, employs a new form of self-conditioning, and applies the diffusion process on the logit simplex space rather than the learned embedding space. Through extensive experiments on natural language understanding and generation tasks including summarization, text simplification, paraphrase generation, and question generation, we demonstrate that TESS outperforms state-of-the-art non-autoregressive models, requires fewer diffusion steps with minimal drop in performance, and is competitive with pretrained autoregressive sequence-to-sequence models. We publicly release our codebase at https://github.com/allenai/tess-diffusion.

Key facts. Cited by 65 papers in the map (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 47 of its 70 references that are in the map. First posted on arXiv 15 May 2023.

Builds on

47 of 70 references.

Closest ancestors · 7

Same direction, newest first

Other references · 40

2017Attention Is All You Needcited by 60,361
2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
2020Denoising Diffusion Probabilistic Modelscited by 21,076
2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
2020Denoising Diffusion Implicit Modelscited by 9,781
2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
2022Classifier-Free Diffusion Guidancecited by 5,847

28 more

2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
2022Video Diffusion Modelscited by 2,305
2023Consistency Modelscited by 1,822
2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,354
2015Teaching Machines to Read and Comprehendcited by 1,049
2022Diffusion-LM Improves Controllable Text Generationcited by 1,027
2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 1,001
2018Neural Network Acceptability Judgmentscited by 958
2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 574
2021Datasets: A Community Library for Natural Language Processingcited by 530
2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 470
2022Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioningcited by 373
2022Efficient Training of Language Models to Fill in the Middlecited by 255
2023NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizerscited by 235
2021Societal Biases in Language Generation: Progress and Challengescited by 202
2019Levenshtein Transformercited by 199
2020Neural CRF Model for Sentence Alignment in Text Simplificationcited by 106
2021Step-unrolled Denoising Autoencoders for Text Generationcited by 102
2018Fast, Diverse and Accurate Image Captioning Guided By Part-of-Speechcited by 55
2017Quasar: Datasets for Question Answering by Search and Readingcited by 50
2022Categorical SDEs with Simplex Diffusioncited by 42
2022Diverse Text Generation via Variational Encoder-Decoder Models with Gaussian Process Priorscited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 6; 2024: 10; 2025: 27; 2026: 22.

65 papers.

Sort:

By area:

2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 745
2024Scaling up Masked Diffusion Models on Textcited by 199
2025LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuningcited by 148
2025A Survey on Latent Reasoningcited by 81
2025A Survey on Diffusion Language Modelscited by 78
2024Fisher Flow Matching for Generative Modeling over Discrete Datacited by 72
2023Bayesian Flow Networkscited by 63
2025Discrete Diffusion in Large Language and Multimodal Models: A Surveycited by 61
2024Categorical Flow Matching on Statistical Manifoldscited by 47
2026ELF: Embedded Language Flowscited by 45
2023DiffuSeq-v2: Bridging Discrete and Continuous Text Spaces for Accelerated Seq2Seq Diffusion Modelscited by 36
2025Continuously Augmented Discrete Diffusion model for Categorical Generative Modelingcited by 34

38 more

2025Coevolutionary Continuous Discrete Diffusion: Make Your Diffusion Language Model a Latent Reasonercited by 32
2025TESS 2: A Large-Scale Generalist Diffusion Language Modelcited by 30
2025LLaDA-VLA: Vision Language Diffusion Action Modelscited by 30
2024Unifying Bayesian Flow Networks and Diffusion Models through Stochastic Differential Equationscited by 28
2025Cosmos: Compressed and Smooth Latent Space for Text Diffusion Modelingcited by 25
2025Diffusion vs. Autoregressive Language Models: A Text Embedding Perspectivecited by 23
2025DLM-One: Diffusion Language Models for One-Step Sequence Generationcited by 20
2023ParaGuide: Guided Diffusion Paraphrasers for Plug-and-Play Textual Style Transfercited by 18
2025α-Flow: A Unified Framework for Continuous-State Discrete Flow Matching Modelscited by 18
2025LLaDA-MedV: Exploring Large Language Diffusion Models for Biomedical Image Understandingcited by 13
2024Diffusion on language model encodings for protein sequence generationcited by 11
2024Glauber Generative Model: Discrete Diffusion Models via Binary Classificationcited by 11
2026How to Train Your Latent Diffusion Language Model Jointly With the Latent Spacecited by 11
2026Posterior Refinement: Fast Language Generation via Any-Order Flow Mapscited by 11
2025Toward Safer Diffusion Language Models: Discovery and Mitigation of Priming Vulnerabilitycited by 10
2024Discrete Diffusion Language Model for Efficient Text Summarizationcited by 8
2025Smoothie: Smoothing Diffusion on Token Embeddings for Text Generationcited by 8
2025MMaDA-Parallel: Multimodal Large Diffusion Language Models for Thinking-Aware Editing and Generationcited by 8
2026IDLM: Inverse-distilled Diffusion Language Modelscited by 8
2025Efficient Parallel Samplers for Recurrent-Depth Models and Their Connection to Diffusion Language Modelscited by 7
2026Streaming-dLLM: Accelerating Diffusion LLMs via Suffix Pruning and Dynamic Decodingcited by 7
2023David helps Goliath: Inference-Time Collaboration Between Small Specialized and Large General Diffusion LMscited by 6
2024Minibatch Optimal Transport and Perplexity Bound Estimation in Discrete Flow Matchingcited by 5
2025TFG-Flow: Training-free Guidance in Multimodal Generative Flowcited by 5
2025SketchDNN: Joint Continuous-Discrete Diffusion for CAD Sketch Generationcited by 5
2025Finish First, Perfect Later: Test-Time Token-Level Cross-Validation for Diffusion Large Language Modelscited by 5
2026Membership Inference Attacks Against Fine-tuned Diffusion Language Modelscited by 5
2026Why Gaussian Diffusion Models Fail on Discrete Data and How to Prevent It?cited by 5
2023P^3SUM: Preserving Author's Perspective in News Summarization with Diffusion Language Modelscited by 4
2025Calibrating Pre-trained Language Classifiers on LLM-generated Noisy Labels via Iterative Refinementcited by 4
2025Unifying Continuous and Discrete Text Diffusion with Non-simultaneous Diffusion Processescited by 4
2025Efficient Perplexity Bound and Ratio Matching in Discrete Diffusion Language Modelscited by 3
2025Simplex-to-Euclidean Bijections for Categorical Flow Matchingcited by 3
2026Dataset-Level Metrics Attenuate Non-Determinism: A Fine-Grained Non-Determinism Evaluation in Diffusion Language Modelscited by 3
2026Forward-Learned Discrete Diffusion: Learning how to noise to denoise fastercited by 3
2024SYMPLEX: Controllable Symbolic Music Generation using Simplex Diffusion with Vocabulary Priorscited by 2
2024Generative Diffusion Models for Sequential Recommendationscited by 2
2025Don't Let It Fade: Preserving Edits in Diffusion Language Models via Token Timestep Allocationcited by 2

50 most cited of 65 All 65 (JSON)

Similar, not cited

Close in wording, no citation either way