mlsanityData to 14 Oct 2026

Nemotron-Labs-TwoTower: Diffusion Language Modeling with Pretrained Autoregressive Context

arXiv 2606.26493 ↗First posted 25 Jun 2026cs.CLcited by 4

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Diffusion language models offer a promising alternative to autoregressive models due to their potential for parallel and iterative generation. However, existing approaches use a single network for both context representation and iterative denoising, forcing one model to serve both roles and limiting its capacity for either role. We propose TwoTower, a block-wise autoregressive diffusion model that decouples these roles into two towers: a frozen AR context tower that causally processes clean tokens, and a trainable diffusion denoiser tower with bidirectional block attention that refines noisy blocks via cross-attention to the context. Built on Nemotron-3-Nano-30B-A3B, an open-weight 30B hybrid Mamba-Transformer MoE model, and trained on approximately 2.1T tokens, Nemotron-Labs-TwoTower retains 98.7% of the autoregressive baseline's quality while offering 2.42X higher wall-clock generation throughput. We release the code and model weights at https://huggingface.co/collections/nvidia/nemotron-labs-twotower.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 13 of its 17 references that are in the map. First posted on arXiv 25 Jun 2026.

Builds on

13 of 17 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Encoder-Decoder Diffusion Language Models for Efficient Training and Inference26 Oct
  2. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models12 Mar
  3. 2025Large Language Diffusion Models14 Feb
  4. 2024Simple and Effective Masked Diffusion Language Models11 Jun
  5. 2024Simplified and Generalized Masked Diffusion for Discrete Data6 Jun
  6. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data6 Jun

Other references · 7

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2022Scalable Diffusion Models with Transformerscited by 6,054
  3. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
  4. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  5. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 864
  6. 2024MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategiescited by 723
  7. 2025NVIDIA Nemotron 3: Efficient and Open Intelligencecited by 65

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026GroupKV: Hierarchical KV Cache Management for Long-Context Diffusion LLM Inferencecited by 2
  2. 2026Context-Tower Conversion Preserves Generation While Freezing Retains Knowledge: Low-Budget AR-to-Diffusion Conversion of MoE LLMscited by 0
  3. 2026SoloQ: Calibration-Free Quantization for Diffusion Language Modelscited by 0
  4. 2026Revisiting the 16x Pretraining Compute Gap between Diffusion and Autoregressive Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Not All Denoising Steps Are Equal: Model Scheduling for Faster Masked Diffusion Language Models
  2. 2025Dream 7B: Diffusion Large Language Models
  3. 2026Nemotron-Labs-Diffusion: A Tri-Mode Language Model Unifying Autoregressive, Diffusion, and Self-Speculation Decoding
  4. 2026DiffusionGemma Technical Report
  5. 2025Esoteric Language Models: A Family of Any-Order Diffusion LLMs
  6. 2025FlashDLM: Accelerating Diffusion Language Model Inference via Efficient KV Caching and Guided Diffusion
  7. 2026LESS Is More: Mutual-Stability Sampling for Diffusion Language Models
  8. 2025A Survey on Diffusion Language Models