mlsanityData to 14 Oct 2026

Next-Concept Modeling: Fast, Parallel Coarse-to-Fine Text Generation

arXiv 2610.14162 ↗First posted 9 Oct 2026cs.CL cs.AI cs.LGcited by 0

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Diffusion language models (DLMs) offer parallel sampling for faster generation than autoregressive models, an increasingly important advantage as inference becomes a dominant computational workload. However, DLMs degrade quickly with fewer sampling steps, process the full sequence at every step, and require many passes over the dataset to reach good quality. We introduce Next-Concept Modeling (NCM), a two-stage, coarse-to-fine framework: a multiscale residual VQ-VAE encodes text into a hierarchy of discrete codes, and a transformer generates one level at a time, predicting all codes within a level in parallel. At matched parameter scales, NCM achieves better generative perplexity than the strongest DLM baseline on LM1B while running 21× faster. On OpenWebText, NCM is competitive with the strongest DLM while running 10× faster; on XSum, it matches the strongest DLM's ROUGE-L while being 31× faster. On the reasoning task of Sudoku, NCM outperforms autoregressive and diffusion baselines across all difficulty levels, using fewer function evaluations. On the reasoning task of Sudoku, NCM outperforms autoregressive and diffusion baselines across all difficulty levels, using fewer function evaluations.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 46 of its 52 references that are in the map. First posted on arXiv 9 Oct 2026.

Builds on

46 of 52 references.

Closest ancestors · 20

Same direction, newest first

  1. 2026AURORA-LM: Autoencoding Unified Representation for Continuous-Latent Diffusion Language Modeling3 Aug
  2. 2026DiffusionGemma Technical Report31 Jul
  3. 2026Language Modeling with Hyperspherical Flows11 May
  4. 2026ELF: Embedded Language Flows11 May
  5. 2026Flow Map Language Models: One-step Language Modeling via Continuous Denoising18 Feb
  6. 2026DFlash: Block Diffusion for Flash Speculative Decoding5 Feb
  7. 2025Encoder-Decoder Diffusion Language Models for Efficient Training and Inference26 Oct
  8. 2025CANDI: Hybrid Discrete-Continuous Diffusion Models26 Oct
  9. 2025Next Semantic Scale Prediction via Hierarchical Diffusion Language Models8 Oct
  10. 2025The Diffusion Duality12 Jun
  11. 2025dKV-Cache: The Cache for Diffusion Language Models21 May
  12. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models12 Mar
8 more
  1. 2025Large Language Diffusion Models14 Feb
  2. 2024Beyond Autoregression: Fast LLMs via Self-Distillation Through Time28 Oct
  3. 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling4 Sep
  4. 2024Simple and Effective Masked Diffusion Language Models11 Jun
  5. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct
  6. 2023PLANNER: Generating Diversified Paragraph via Latent Language Diffusion Model5 Jun
  7. 2022SeqDiffuSeq: Text Diffusion with Encoder-Decoder Transformers20 Dec
  8. 2022Latent Diffusion for Language Generation19 Dec

Other references · 26

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  2. 2017Neural Discrete Representation Learningcited by 4,712
  3. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  4. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  5. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  6. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,387
  7. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,317
  8. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,023
  9. 2021SoundStream: An End-to-End Neural Audio Codeccited by 771
  10. 2022Autoregressive Image Generation using Residual Quantizationcited by 722
  11. 2022High Fidelity Neural Audio Compressioncited by 687
  12. 2022Efficiently Scaling Transformer Inferencecited by 539
14 more
  1. 2023TinyStories: How Small Can Language Models Be and Still Speak Coherent English?cited by 449
  2. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 420
  3. 2013One Billion Word Benchmark for Measuring Progress in Statistical Language Modelingcited by 308
  4. 2025Mercury: Ultra-Fast Language Models Based on Diffusioncited by 235
  5. 2023MEGABYTE: Predicting Million-byte Sequences with Multiscale Transformerscited by 170
  6. 2025Fast-dLLM v2: Efficient Block-Diffusion LLMcited by 139
  7. 2024ImageFolder: Autoregressive Image Generation with Folded Tokenscited by 94
  8. 2024Large Concept Models: Language Modeling in a Sentence Representation Spacecited by 89
  9. 2025Dynamic Chunking for End-to-End Hierarchical Sequence Modelingcited by 72
  10. 2023It's MBR All the Way Down: Modern Generation Techniques Through the Lens of Minimum Bayes Riskcited by 51
  11. 2025HMAR: Efficient Hierarchical Masked Auto-Regressive Image Generationcited by 20
  12. 2025Dynamic Large Concept Models: Latent Reasoning in an Adaptive Semantic Spacecited by 17
  13. 2026Next Concept Prediction in Discrete Latent Space Leads to Stronger Language Modelscited by 11
  14. 2026Posterior Refinement: Fast Language Generation via Any-Order Flow Mapscited by 11

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Diffusion Language Models: An Experimental Analysis
  2. 2023Diffusion Language Models Generation Can Be Halted Early
  3. 2025FS-DFM: Fast and Accurate Long Text Generation with Few-Step Diffusion Language Models
  4. 2025A Survey on Parallel Text Generation: From Parallel Decoding to Diffusion Language Models
  5. 2025Theoretical Benefit and Limitation of Diffusion Language Model
  6. 2026Hierarchical Continuous Diffusion Language Models
  7. 2026From Position Risks to Block Survival: Faster Generation for Diffusion Language Models
  8. 2025DLM-One: Diffusion Language Models for One-Step Sequence Generation