mlsanityData to 14 Oct 2026

Spec-LLaVA: Accelerating Vision-Language Models with Dynamic Tree-Based Speculative Decoding

arXiv 2509.11961 ↗First posted 15 Sep 2025cs.CLcited by 9

On the map: LLM reasoning → Test-time compute and efficient thinking

also in 2 more
Abstract (arXiv)

Vision-Language Models (VLMs) enable powerful multimodal reasoning but suffer from slow autoregressive inference, limiting their deployment in real-time applications. We introduce Spec-LLaVA, a system that applies speculative decoding to accelerate VLMs without sacrificing output quality. Spec-LLaVA pairs a lightweight draft VLM with a large target model: the draft speculates future tokens, which the target verifies in parallel, allowing multiple tokens to be generated per step. To maximize efficiency, we design a dynamic tree-based verification algorithm that adaptively expands and prunes speculative branches using draft model confidence. On MS COCO out-of-domain images, Spec-LLaVA achieves up to 3.28× faster decoding on LLaVA-1.5 (7B, 13B) with no loss in generation quality. This work presents a lossless acceleration framework for VLMs using dynamic tree-structured speculative decoding, opening a path toward practical real-time multimodal assistants. Importantly, the lightweight draft model design makes the framework amenable to resource-constrained or on-device deployment settings.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 28 of its 36 references that are in the map. First posted on arXiv 15 Sep 2025.

Builds on

28 of 36 references.

References · 28

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  3. 2023Visual Instruction Tuningcited by 7,869
  4. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
  5. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
  6. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 1,006
  7. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 843
  8. 2024EAGLE-2: Faster Inference of Language Models with Dynamic Draft Treescited by 370
  9. 2024Hydra: Sequentially-Dependent Draft Heads for Medusa Decodingcited by 117
  10. 2024On-Device Language Models: A Comprehensive Reviewcited by 104
  11. 2024Sequoia: Scalable, Robust, and Hardware-aware Speculative Decodingcited by 103
  12. 2021Consistent Accelerated Inference via Confident Adaptive Transformerscited by 74
16 more
  1. 2024OPT-Tree: Speculative Decoding with Adaptive Draft Tree Structurecited by 67
  2. 2023PaSS: Parallel Speculative Samplingcited by 49
  3. 2024MENTOR: Multi-level Self-supervised Learning for Multimodal Recommendationcited by 21
  4. 2025SETransformer: A Hybrid Attention-Based Architecture for Robust Human Activity Recognitioncited by 19
  5. 2025MonoFusion: Sparse-View 4D Reconstruction via Monocular Fusioncited by 19
  6. 2024BiTA: Bi-Directional Tuning for Lossless Acceleration in Large Language Modelscited by 13
  7. 2023Human-oriented Representation Learning for Robotic Manipulationcited by 10
  8. 2024Joint Pedestrian Trajectory Prediction through Posterior Samplingcited by 10
  9. 2025COHESION: Composite Graph Convolutional Network with Dual-Stage Fusion for Multimodal Recommendationcited by 10
  10. 2023An Integrated Transportation Distance Between Kernels and Approximate Dynamic Risk Evaluation in Markov Systemscited by 9
  11. 2024Impact of Noisy Supervision in Foundation Model Learningcited by 8
  12. 2022AbHE: All Attention-based Homography Estimationcited by 7
  13. 2023Fast Dual Subgradient Optimization of the Integrated Transportation Distance Between Stochastic Kernelscited by 4
  14. 2024Don't Lose Yourself: Boosting Multimodal Recommendation via Reducing Node-neighbor Discrepancy in Graph Convolutional Networkcited by 2
  15. 2025Leveraging Optimal Transport for Distributed Two-Sample Testing: An Integrated Transportation Distance-based Frameworkcited by 1
  16. 2025Federated Calculation of the Free-Support Transportation Barycenter by Single-Loop Dual Decompositioncited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 9.

9 papers.

  1. 2026Efficient Inference for Large Vision-Language Models: Bottlenecks, Techniques, and Prospectscited by 9
  2. 2026HSD: Training-Free Acceleration for Document Parsing Vision-Language Models with Hierarchical Speculative Decodingcited by 6
  3. 2026See the Forest for the Trees: Loosely Speculative Decoding via Visual-Semantic Guidance for Efficient Inference of Video LLMscited by 5
  4. 2026Sparrow: Text-Anchored Window Attention with Visual-Semantic Glimpsing for Speculative Decoding in Video LLMscited by 4
  5. 2026AsymSpec: Context-Asymmetric Speculative Decoding for Agentic LLMscited by 1
  6. 2026Deltoris: Enabling Real-time VLA Inference in Embodied AI via Bit-level Sparsity and Speculative Inferencecited by 0
  7. 2026Is Multimodal Speculative Decoding Ready for Diffusion-Based Parallel Drafting? A Survey and Empirical Diagnosiscited by 0
  8. 2026Vision Is Not Overhead: One-Pass Block Drafting for Lossless Speculative Decoding in Vision-Language Modelscited by 0
  9. 2026SP2ec: Adaptive Self-Speculative Decoding for Vision-Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025SpecVLM: Fast Speculative Decoding in Vision-Language Models
  2. 2026MMSpec: Benchmarking Speculative Decoding for Vision-Language Models
  3. 2025ViSpec: Accelerating Vision-Language Models with Vision-Aware Speculative Decoding
  4. 2026SAGE: Accelerating Vision-Language Models via Entropy-Guided Adaptive Speculative Decoding
  5. 2025MASSV: Multimodal Adaptation and Self-Data Distillation for Speculative Decoding of Vision-Language Models
  6. 2024On Speculative Decoding for Multimodal Large Language Models
  7. 2025FastVLM: Self-Speculative Decoding for Fast Vision-Language Model Inference
  8. 2025Spec-VLA: Speculative Decoding for Vision-Language-Action Models with Relaxed Acceptance