mlsanityData to 7 Oct 2026

Diffusion Drafts, AR Verifies: Accelerating Document OCR with Self-Speculative Decoding

arXiv 2609.26638 ↗First posted 22 Sep 2026cs.CL cs.CVcited by 0

On the map: Efficient LLM inference and serving → Fast autoregressive visual generation

Abstract (arXiv)

Autoregressive OCR vision-language models accurately convert document images into text and structured markup, but require one sequential decoding step per output token, limiting inference speed. Unlike open-ended text generation, OCR outputs are strongly grounded in the input image, making diffusion-based parallel generation promising. However, when several tokens are predicted in one diffusion step, each is predicted before the others are known. Committing them directly can therefore introduce errors. We therefore introduce GravityOCR, a parameter-shared AR-block-diffusion model jointly trained for parallel drafting and causal AR verification. Verifying drafts before commitment lets the model commit multiple output tokens per round without a separate drafting network. The causal AR path also enables GRPO with sequence- and structure-level OCR rewards, avoiding diffusion-trajectory likelihood estimation while updating the shared drafter parameters. On OmniDocBench v1.6, AR-path GRPO improves the Overall score from 94.92 to 95.16 without reducing diffusion drafting efficiency, while the final model remains close to the original GLM-OCR score of 95.48. In an SGLang serving deployment, GravityOCR commits an average of 9.7 output tokens per forward pass and achieves a 3.94× decode-only speedup on region crops and a 1.32× end-to-end page-processing speedup over AR decoding.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Fast autoregressive visual generation, one of 12 directions in Efficient LLM inference and serving. Builds on 55 of its 57 references that are in the map. First posted on arXiv 22 Sep 2026.

Builds on

55 of 57 references.

Closest ancestors · 1

Same direction, newest first

  1. 2026HSD: Training-Free Acceleration for Document Parsing Vision-Language Models with Hierarchical Speculative Decoding13 Feb

Other references · 54

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  3. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  4. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  5. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
  6. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 975
  7. 2025Large Language Diffusion Modelscited by 872
  8. 2024Simple and Effective Masked Diffusion Language Modelscited by 800
  9. 2025Dream 7B: Diffusion Large Language Modelscited by 515
  10. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 404
  11. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Modelscited by 385
  12. 2023Nougat: Neural Optical Understanding for Academic Documentscited by 240
38 more
  1. 2019Image-based table recognition: data, model, and evaluationcited by 233
  2. 2015Evaluation of Deep Convolutional Nets for Document Image Classification and Retrievalcited by 212
  3. 2025d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learningcited by 178
  4. 2025DeepSeek-OCR: Contexts Optical Compressioncited by 174
  5. 2024Building and better understanding vision-language models: insights and future directionscited by 169
  6. 2020Global Table Extractor (GTE): A Framework for Joint Table Identification and Cell Structure Recognition Using Visual Contextcited by 146
  7. 2022DocLayNet: A Large Human-Annotated Dataset for Document-Layout Analysiscited by 146
  8. 2024OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotationscited by 144
  9. 2025Fast-dLLM v2: Efficient Block-Diffusion LLMcited by 128
  10. 2021PubTables-1M: Towards comprehensive table extraction from unstructured documentscited by 122
  11. 2025olmOCR: Unlocking Trillions of Tokens in PDFs with Vision Language Modelscited by 120
  12. 2026DFlash: Block Diffusion for Flash Speculative Decodingcited by 120
  13. 2025SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generationcited by 118
  14. 2025MinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document Parsingcited by 108
  15. 2022TableFormer: Table Structure Understanding with Transformerscited by 93
  16. 2025PaddleOCR-VL: Boosting Multilingual Document Parsing via a 0.9B Ultra-Compact Vision-Language Modelcited by 84
  17. 2026DeepSeek-OCR 2: Visual Causal Flowcited by 68
  18. 2024Speculative Diffusion Decoding: Accelerating Language Generation through Diffusioncited by 57
  19. 2026PaddleOCR-VL-1.5: Towards a Multi-Task 0.9B VLM for Robust In-the-Wild Document Parsingcited by 54
  20. 2025dots.ocr: Multilingual Document Layout Parsing in a Single Vision-Language Modelcited by 44
  21. 2024UniMERNet: A Universal Network for Real-World Mathematical Expression Recognitioncited by 41
  22. 2025DiffuSpec: Unlocking Diffusion Language Models for Speculative Decodingcited by 40
  23. 2024Image Over Text: Transforming Formula Recognition Evaluation with Character Detection Matchingcited by 39
  24. 2025WeDLM: Reconciling Diffusion Language Models with Standard Causal Attention for Fast Inferencecited by 38
  25. 2025TiDAR: Think in Diffusion, Talk in Autoregressioncited by 35
  26. 2024DocGenome: An Open Large-scale Scientific Document Benchmark for Training and Testing Multi-modal Large Language Modelscited by 34
  27. 2025PP-DocLayout: A Unified Document Layout Detection Model to Accelerate Large-Scale Data Constructioncited by 26
  28. 2025SpecDiff-2: Scaling Diffusion Drafter Alignment For Faster Speculative Decodingcited by 26
  29. 2026MinerU2.5-Pro: Pushing the Limits of Data-Centric Document Parsing at Scalecited by 26
  30. 2026GLM-OCR Technical Reportcited by 24
  31. 2026Nemotron-Labs-Diffusion: A Tri-Mode Language Model Unifying Autoregressive, Diffusion, and Self-Speculation Decodingcited by 19
  32. 2026D-PACE: Dynamic Position-Aware Cross-Entropy for Parallel Speculative Draftingcited by 16
  33. 2026HunyuanOCR-1.5: Making Lightweight OCR VLMs Faster and Bettercited by 14
  34. 2025Simple Policy Gradients for Reasoning with Diffusion Language Modelscited by 9
  35. 2026Fast-dVLM: Efficient Block-Diffusion VLM via Direct Conversion from Autoregressive VLMcited by 8
  36. 2026MinerU-Diffusion: Rethinking Document OCR as Inverse Rendering via Diffusion Decodingcited by 7
  37. 2026EfficientRollout: System-Aware Self-Speculative Decoding for RL Rolloutscited by 5
  38. 2026OvisOCR2 Technical Reportcited by 5

50 most cited of 54 All 54 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Hawk: Leveraging Spatial Context for Faster Autoregressive Text-to-Image Generation
  2. 2023PanGu-Draw: Advancing Resource-Efficient Text-to-Image Synthesis with Time-Decoupled Training and Reusable Coop-Diffusion
  3. 2026DharmaOCR: Specialized Small Language Models for Structured OCR that outperform Open-Source and Commercial Baselines
  4. 2023Accelerating Text-to-Image Editing via Cache-Enabled Sparse Diffusion Inference
  5. 2023DECDM: Document Enhancement using Cycle-Consistent Diffusion Models
  6. 2026PRESTO: Prefix-Aligned Tree Drafting for Diffusion Speculative Decoding
  7. 2026Where Vision Becomes Text: Locating the OCR Routing Bottleneck in Vision-Language Models
  8. 2025Progress by Pieces: Test-Time Scaling for Autoregressive Image Generation