mlsanityData to 14 Oct 2026

TiDAR: Think in Diffusion, Talk in Autoregression

arXiv 2511.08923 ↗First posted 12 Nov 2025cs.CL cs.AIcited by 37

On the map: LLM reasoning → Diffusion reasoning and test-time scaling

also in 2 more
Abstract (arXiv)

Diffusion language models hold the promise of fast parallel generation, while autoregressive (AR) models typically excel in quality due to their causal structure aligning naturally with language modeling. This raises a fundamental question: can we achieve a synergy with high throughput, higher GPU utilization, and AR level quality? Existing methods fail to effectively balance these two aspects, either prioritizing AR using a weaker model for sequential drafting (speculative decoding), leading to lower drafting efficiency, or using some form of left-to-right (AR-like) decoding logic for diffusion, which still suffers from quality degradation and forfeits its potential parallelizability. We introduce TiDAR, a sequence-level hybrid architecture that drafts tokens (Thinking) in Diffusion and samples final outputs (Talking) AutoRegressively - all within a single forward pass using specially designed structured attention masks. This design exploits the free GPU compute density, achieving a strong balance between drafting and verification capacity. Moreover, TiDAR is designed to be serving-friendly (low overhead) as a standalone model. We extensively evaluate TiDAR against AR models, speculative decoding, and diffusion variants across generative and likelihood tasks at 1.5B and 8B scales. Thanks to the parallel drafting and sampling as well as exact KV cache support, TiDAR outperforms speculative decoding in measured throughput and surpasses diffusion models like Dream and Llada in both efficiency and quality. Most notably, TiDAR is the first architecture to close the quality gap with AR models while delivering 4.71x to 5.91x more tokens per second.

Key facts. Cited by 37 papers in the map (data through 14 Oct 2026). Placed in Diffusion reasoning and test-time scaling, one of 17 directions in LLM reasoning. Builds on 24 of its 52 references that are in the map. First posted on arXiv 12 Nov 2025.

Builds on

24 of 52 references.

References · 24

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  4. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  5. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  6. 2021Program Synthesis with Large Language Modelscited by 3,637
  7. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  8. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 1,006
  9. 2025Large Language Diffusion Modelscited by 912
  10. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 843
  11. 2024DeepSeek-V3 Technical Reportcited by 811
  12. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertaintycited by 578
12 more
  1. 2025Dream 7B: Diffusion Large Language Modelscited by 544
  2. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Modelscited by 409
  3. 2024EAGLE-2: Faster Inference of Language Models with Dynamic Draft Treescited by 370
  4. 2024Better & Faster Large Language Models via Multi-token Predictioncited by 360
  5. 2025SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Modelcited by 335
  6. 2025EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Testcited by 331
  7. 2024Flex Attention: A Programming Model for Generating Optimized Attention Kernelscited by 185
  8. 2024LLM Inference Unveiled: Survey and Roofline Model Insightscited by 152
  9. 2025Accelerating Diffusion LLMs via Adaptive Parallel Decodingcited by 112
  10. 2025Theoretical Benefit and Limitation of Diffusion Language Modelcited by 47
  11. 2025Set Block Decoding is a Language Model Inference Acceleratorcited by 23
  12. 2025Sequential Diffusion Language Modelscited by 19

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 34.

37 papers.

Sort:

By area:

  1. 2025LLaDA2.0: Scaling Up Diffusion Language Models to 100Bcited by 166
  2. 2026LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decodingcited by 28
  3. 2026Scaling Beyond Masked Diffusion Language Modelscited by 27
  4. 2026Nemotron-Labs-Diffusion: A Tri-Mode Language Model Unifying Autoregressive, Diffusion, and Self-Speculation Decodingcited by 24
  5. 2025CDLM: Consistency Diffusion Language Models For Faster Samplingcited by 21
  6. 2025Dream-VL & Dream-VLA: Open Vision-Language and Vision-Language-Action Models with Diffusion Language Model Backbonecited by 18
  7. 2026DMax: Aggressive Parallel Decoding for dLLMscited by 18
  8. 2026dLLM: Simple Diffusion Language Modelingcited by 14
  9. 2026CoDAR: Continuous Diffusion Language Models are More Powerful Than You Thinkcited by 14
  10. 2026Stable-DiffCoder: Pushing the Frontier of Code Diffusion Large Language Modelcited by 13
  11. 2026Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformerscited by 13
  12. 2026Parallelism and Generation Order in Masked Diffusion Language Models: Limits Today, Potential Tomorrowcited by 11
25 more
  1. 2026Fast-dVLM: Efficient Block-Diffusion VLM via Direct Conversion from Autoregressive VLMcited by 9
  2. 2026DFlare: Scaling Up Draft Capacity for Block Diffusion Speculative Decodingcited by 9
  3. 2026Introspective Diffusion Language Modelscited by 8
  4. 2026FLARE: Diffusion for Hybrid Language Modelcited by 6
  5. 2026Revealing the Attention Floating Mechanism in Masked Diffusion Modelscited by 5
  6. 2026TreeFlash: Parallel AR-Approximation for Faster Speculative Decodingcited by 4
  7. 2026DeLS-Spec: Decoupled Long-Short Contexts for Parallel Speculative Draftingcited by 4
  8. 2026dTRPO: Trajectory Reduction in Policy Optimization of Diffusion Large Language Modelscited by 3
  9. 2026Locally Coherent Parallel Decoding in Diffusion Language Modelscited by 3
  10. 2026Dataset-Level Metrics Attenuate Non-Determinism: A Fine-Grained Non-Determinism Evaluation in Diffusion Language Modelscited by 3
  11. 2026TRIMS: Trajectory-Ranked Instruction Masked Supervision for Diffusion Language Modelscited by 2
  12. 2026FlexDraft: Flexible Speculative Decoding via Attention Tuning and Bonus-Guided Calibrationcited by 2
  13. 2026Bastion: Budget-Aware Speculative Decoding with Tree-structured Block Diffusion Draftingcited by 2
  14. 2026Unlocking Lossless Speedups in LLMs via Discrete Diffusioncited by 2
  15. 2026Projected Autoregression: Autoregressive Language Generation in Continuous State Spacecited by 1
  16. 2026Tuning the Implicit Regularizer of Masked Diffusion Language Models: Enhancing Generalization via Insights from k-Paritycited by 1
  17. 2026Enabling KV Caching of Shared Prefix for Diffusion Language Modelscited by 1
  18. 2026From Interface to Inference: Eliciting Any-Order Inference from Any-Order Modelscited by 1
  19. 2026LEAP: Unlocking dLLM Parallelism via Lookahead Early-Convergence Token Detectioncited by 0
  20. 2026BlockGen: Flexible Blockwise Sequence Modeling with Hybrid Samplerscited by 0
  21. 2026Bifocal Diffusion Language Models: Asymmetric Bidirectional Context for Parallel Generationcited by 0
  22. 2026Diffusion Drafts, AR Verifies: Accelerating Document OCR with Self-Speculative Decodingcited by 0
  23. 2026Reciprocal Guidance: Orchestrating Draft and Verify Budgets for Advancing the Diffusion-AR Self-Speculation Frontiercited by 0
  24. 2026SEED: Self-Speculative Decoding via Implicit Encoder-Decodercited by 0
  25. 2026ALoDLM: Adaptively Looped Diffusion Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Speculative Refinement: A Hybrid Autoregressive Diffusion Decoding Strategy and Its Behavior Across Benchmarks
  2. 2025Diffusion Beats Autoregressive in Data-Constrained Settings
  3. 2025SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generation
  4. 2025On Powerful Ways to Generate: Autoregression, Diffusion, and Beyond
  5. 2025Esoteric Language Models: A Family of Any-Order Diffusion LLMs
  6. 2026SimSD: Simple Speculative Decoding in Diffusion Language Models
  7. 2026Zarya: A Hybrid Autoregressive--Masked Diffusion Language Model with Flexible Training and Dual-Mode Inference
  8. 2026Autoregressive vs. Masked Diffusion Language Models: A Controlled Comparison