mlsanityData to 7 Oct 2026

LumoTree: Path-Parallel Speculative Verification for Hybrid Language Models

arXiv 2609.23900 ↗First posted 20 Sep 2026cs.LGcited by 0

On the map: Efficient LLM inference and serving → Speculative decoding

Abstract (arXiv)

Tree speculative decoding for recurrent-hybrid language models requires each accepted path to maintain consistent recurrent state, convolution history, and attention caches. We present LumoTree, a GPU serving design that coordinates verification and commitment through a shared tree descriptor. The verifier processes independent paths in parallel and reuses recurrent state tiles across local updates. Boundary states connect dependent paths. Accepted-path replay publishes the selected continuation into native running state, alongside coordinated convolution gathering and attention-cache remapping. Fused selection, device-resident acceptance, graph replay, and tree-aware attention integrate this organization into the serving cycle. The design separates temporary branch computation from persistent request state while retaining native prefix-cache interfaces. We evaluate numerical behavior, serving performance, and coding-agent outcomes on NVIDIA DGX Spark.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving. Builds on 26 of its 37 references that are in the map. First posted on arXiv 20 Sep 2026.

Builds on

26 of 37 references.

Closest ancestors · 16

Same direction, newest first

  1. 2026TreeWY: Speculative Verification for Gated DeltaNet Hybrids21 Aug
  2. 2026Bole: Efficient Tree Speculation for Hybrid-Attention Language Models3 Aug
  3. 2026SpecLA: Efficient Speculative Decoding for Linear-Attention Models18 Jul
  4. 2026Trees from Marginals: Autoregressive drafting with factorized priors7 Jul
  5. 2026Component-Aware Self-Speculative Decoding in Hybrid Language Models1 May
  6. 2026DFlash: Block Diffusion for Flash Speculative Decoding5 Feb
  7. 2025SpecMamba: Accelerating Mamba Inference on FPGA with Speculative Decoding24 Sep
  8. 2025STree: Speculative Tree Decoding for Hybrid State-Space Models20 May
  9. 2025EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test3 Mar
  10. 2024SAM Decoding: Speculative Decoding via Suffix Automaton16 Nov
  11. 2024SuffixDecoding: Extreme Speculative Decoding for Emerging AI Applications7 Nov
  12. 2024OPT-Tree: Speculative Decoding with Adaptive Draft Tree Structure25 Jun
4 more
  1. 2024EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees24 Jun
  2. 2023SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification16 May
  3. 2023Accelerating Large Language Model Decoding with Speculative Sampling2 Feb
  4. 2022Fast Inference from Transformers via Speculative Decoding30 Nov

Other references · 10

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  2. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  3. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  4. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
  5. 2024Gated Delta Networks: Improving Mamba2 with Delta Rulecited by 483
  6. 2024Parallelizing Linear Transformers with the Delta Rule over Sequence Lengthcited by 373
  7. 2025FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Servingcited by 248
  8. 2024Marconi: Prefix Caching for the Era of Hybrid LLMscited by 29
  9. 2024DeFT: Decoding with Flash Tree-attention for Efficient Tree-structured LLM Inferencecited by 19
  10. 2026OneLA: Scaling Linear-Attention Decoding to Large Beams in Generative Recommendationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026CATS: Cascaded Adaptive Tree Speculation for Memory-Limited LLM Inference Acceleration
  2. 2026CAST: Cost-Aware Speculative Trees from One-Pass Block Drafters
  3. 2026Bastion: Budget-Aware Speculative Decoding with Tree-structured Block Diffusion Drafting
  4. 2026DScale: Scaling Block-Diffusion Speculative Decoding with Adaptive Verification
  5. 2024Clover: Regressive Lightweight Speculative Decoding with Sequential Knowledge
  6. 2026EAGLE-Pangu: Accelerator-Safe Tree Speculative Decoding on Ascend NPUs
  7. 2025Speculative Verification: Exploiting Information Gain to Refine Speculative Decoding
  8. 2026ECHO: Elastic Speculative Decoding with Sparse Gating for High-Concurrency Scenarios