mlsanityData to 14 Oct 2026

ConfigSpec: Profiling-Based Configuration Selection for Distributed Edge--Cloud Speculative LLM Serving

arXiv 2604.09722 ↗First posted 8 Apr 2026cs.DC cs.AIcited by 1

On the map: Efficient LLM inference and serving → Early exit and edge inference

Abstract (arXiv)

Speculative decoding enables collaborative Large Language Model (LLM) inference across cloud and edge by separating lightweight token drafting from heavyweight verification. While prior systems show performance and cost benefits, practical deployment requires navigating a large configuration space spanning draft model variants, quantisation levels, speculative lengths, and heterogeneous edge devices. This paper presents ConfigSpec, a configurationselection framework for distributed speculative LLM serving. ConfigSpec profiles edge devices and draft-target alignment, and models drafting throughput, acceptance rate, and power to evaluate goodput, verification cost efficiency, and energy efficiency across the joint configuration space. Our analysis across three edge platforms and two LLM families reveals structurally conflicting optima. Firstly, goodput is maximised by the smallest, fastest draft model at device-dependent speculative lengths (K*=2-10). Secondly, both cost and energy efficiency converge to K=2 due to a dominant bonus-token effect-with cost favouring the largest drafter for its high acceptance rate and energy favouring the smallest for its low power draw. These conflicts confirm that no single fixed configuration can simultaneously optimise all objectives, underscoring the need for profiling-based configuration selection in disaggregated edge-cloud LLM inference.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Early exit and edge inference, one of 12 directions in Efficient LLM inference and serving. Builds on 8 of its 17 references that are in the map. First posted on arXiv 8 Apr 2026.

Builds on

8 of 17 references.

References · 8

  1. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
  2. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,121
  3. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 1,006
  4. 2024EfficientQAT: Efficient Quantization-Aware Training for Large Language Modelscited by 127
  5. 2024MagicDec: Breaking the Latency-Throughput Tradeoff for Long Context Generation with Speculative Decodingcited by 91
  6. 2024EDGE-LLM: Enabling Efficient Large Language Model Adaptation on Edge Devices via Layerwise Unified Compression and Adaptive Layer Tuning and Votingcited by 27
  7. 2025CLONE: Customizing LLMs for Efficient Latency-Aware Inference at the Edgecited by 16
  8. 2025SLED: A Speculative LLM Decoding Framework for Efficient Edge Servingcited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Delay-Adaptive Speculation Control for Low-Latency Edge-Cloud LLM Inferencecited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026WISP: Waste- and Interference-Suppressed Distributed Speculative LLM Serving at the Edge via Dynamic Drafting and SLO-Aware Batching
  2. 2026SPADE: Speculative Decoding for Precise and Low Cost Distributed Edge Cloud Inference
  3. 2026A Pipelined Collaborative Speculative Decoding Framework for Efficient Edge-Cloud LLM Inference
  4. 2026AceSpec: An Asymmetric Edge-Cloud Collaborative Framework for Communication-Efficient LLM Inference
  5. 2025DSD: A Distributed Speculative Decoding Solution for Edge-Cloud Agile Large Model Serving
  6. 2025SpecEdge: Scalable Edge-Assisted Serving Framework for Interactive LLMs
  7. 2026WANSpec: Leveraging Global Compute Capacity for LLM Inference
  8. 2026FlexSpec: Frozen Drafts Meet Evolving Targets in Edge-Cloud Collaborative LLM Speculative Decoding