mlsanityData to 14 Oct 2026

Collaborative Large Language Model Inference via Resource-Aware Parallel Speculative Decoding

arXiv 2511.01695 ↗First posted 3 Nov 2025cs.LG eess.SPcited by 2

On the map: Efficient LLM inference and serving → Speculative decoding

Abstract (arXiv)

The growing demand for on-device large language model (LLM) inference highlights the need for efficient mobile edge computing (MEC) solutions, especially in resource-constrained settings. Speculative decoding offers a promising solution by partitioning token generation between a lightweight draft model on mobile devices and a powerful target model on edge servers, but suffers from communication overhead and asynchronous delays. This paper is the first to propose a unified framework that jointly optimizes user association and resource allocation (UARA) to support efficient parallel speculative decoding. We solve the UARA problem using a multi-agent deep reinforcement learning algorithm. To evaluate our approach under realistic conditions, we conduct experiments using the Sionna simulator. Results show that our method achieves up to 28.0% and an average of 23.7% reduction in end-to-end latency without compromising inference accuracy, enabling scalable and low-latency LLM services in MEC systems.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving. Builds on 12 of its 21 references that are in the map. First posted on arXiv 3 Nov 2025.

Builds on

12 of 21 references.

Closest ancestors · 7

Same direction, newest first

  1. 2024PEARL: Parallel Speculative Decoding with Adaptive Draft Length13 Aug
  2. 2024Break the Sequential Dependency of LLM Inference Using Lookahead Decoding3 Feb
  3. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads19 Jan
  4. 2023DistillSpec: Improving Speculative Decoding via Knowledge Distillation12 Oct
  5. 2023SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification16 May
  6. 2023Accelerating Large Language Model Decoding with Speculative Sampling2 Feb
  7. 2022Fast Inference from Transformers via Speculative Decoding30 Nov

Other references · 5

  1. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  2. 2022Sionna: An Open-Source Library for Next-Generation Physical Layer Researchcited by 145
  3. 2024CE-CoLLM: Efficient and Adaptive Large Language Models Through Cloud-Edge Collaborationcited by 25
  4. 2023Real-time Neural Network Inference on Extremely Weak Devices: Agile Offloading with Explainable AIcited by 17
  5. 2024Distributed Task Offloading and Resource Allocation for Latency Minimization in Mobile Edge Computing Networkscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026GELATO: Generative Entropy- and Lyapunov-based Adaptive Token Offloading for Device-Edge Speculative LLM Inferencecited by 2
  2. 2026Speculation at a Distance: Where Edge-Cloud Speculative Decoding Actually Pays Offcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026BALANCE: Hybrid Autoregressive-Speculative LLM Inference at the Network Edge
  2. 2026MemSpec: Memory-Aware Runtime for Adaptive Draft Scheduling in Speculative Decoding on Edge Devices
  3. 2025SLED: A Speculative LLM Decoding Framework for Efficient Edge Serving
  4. 2026A Pipelined Collaborative Speculative Decoding Framework for Efficient Edge-Cloud LLM Inference
  5. 2026FlexSpec: Frozen Drafts Meet Evolving Targets in Edge-Cloud Collaborative LLM Speculative Decoding
  6. 2025GoodSpeed: Optimizing Fair Goodput with Adaptive Speculative Decoding in Distributed Edge Inference
  7. 2025MNN-LLM: A Generic Inference Engine for Fast Large Language Model Deployment on Mobile Devices
  8. 2026AgentSpec: Speculative Decoding for Batch Inference of LLM Agents