mlsanityData to 14 Oct 2026

AceSpec: An Asymmetric Edge-Cloud Collaborative Framework for Communication-Efficient LLM Inference

arXiv 2609.02514 ↗First posted 2 Sep 2026cs.DCcited by 0

On the map: Efficient LLM inference and serving → Early exit and edge inference

Abstract (arXiv)

Deploying Large Language Models (LLMs) on edge devices typically relies on model compression or split inference. However, compression degrades reasoning capabilities, while split inference suffers from severe Wide Area Network (WAN) communication bottlenecks. Edge-cloud speculative decoding emerges as a promising alternative, leveraging an edge small model to draft tokens for cloud verification. Yet, over volatile WANs, inevitable prediction rejections trigger catastrophic pipeline stalls and network-wide rollbacks, neutralizing collaborative gains. To overcome this, we propose AceSpec, an asymmetric edge-cloud collaborative framework. AceSpec utilizes un-saturated edge compute to proactively construct a probabilistic state cache, effectively transforming network-wide pipeline flushes into O(1) local memory lookups. To preserve bandwidth, it employs an asymmetric communication protocol that transmits minimal main-chain indices uplink and compact sparse distributions downlink. Furthermore, we introduce a network-aware, Lagrangian-optimized resource allocation strategy that dynamically maximizes the local cache hit rate. Evaluations demonstrate that AceSpec achieves up to a 3.52× throughput speedup and exhibits exceptional bandwidth immunity, sustaining near-peak inference performance even under severely constrained 50 Kbps WAN conditions.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Early exit and edge inference, one of 12 directions in Efficient LLM inference and serving. Builds on 18 of its 25 references that are in the map. First posted on arXiv 2 Sep 2026.

Builds on

18 of 25 references.

Closest ancestors · 6

Same direction, newest first

  1. 2026PipeSD: An Efficient Cloud-Edge Collaborative Pipeline Inference Framework with Speculative Decoding13 May
  2. 2026A Pipelined Collaborative Speculative Decoding Framework for Efficient Edge-Cloud LLM Inference19 Mar
  3. 2025DSD: A Distributed Speculative Decoding Solution for Edge-Cloud Agile Large Model Serving26 Nov
  4. 2025Fast and Cost-effective Speculative Edge-Cloud Decoding with Early Exits27 May
  5. 2025Jupiter: Fast and Resource-Efficient Collaborative Inference of Generative LLMs on Edge Devices11 Apr
  6. 2025A Novel Hat-Shaped Device-Cloud Collaborative Inference Framework for Large Language Models23 Mar

Other references · 12

  1. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 1,006
  2. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 843
  3. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertaintycited by 578
  4. 2024Hybrid LLM: Cost-Efficient and Quality-Aware Query Routingcited by 372
  5. 2023SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verificationcited by 365
  6. 2024A Comprehensive Survey of Small Language Models in the Era of Large Language Models: Techniques, Enhancements, Applications, Collaboration with LLMs, and Trustworthinesscited by 127
  7. 2024A Review on Edge Large Language Models: Design, Execution, and Applicationscited by 100
  8. 2024PEARL: Parallel Speculative Decoding with Adaptive Draft Lengthcited by 85
  9. 2025DSSD: Efficient Edge-Device LLM Deployment and Collaborative Inference via Distributed Split Speculative Decodingcited by 15
  10. 2026Speculative Speculative Decodingcited by 15
  11. 2025SLED: A Speculative LLM Decoding Framework for Efficient Edge Servingcited by 13
  12. 2025Collaborative Speculative Inference for Efficient LLM Inference Servingcited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026FlexSpec: Frozen Drafts Meet Evolving Targets in Edge-Cloud Collaborative LLM Speculative Decoding
  2. 2026ConfigSpec: Profiling-Based Configuration Selection for Distributed Edge--Cloud Speculative LLM Serving
  3. 2024EdgeShard: Efficient LLM Inference via Collaborative Edge Computing
  4. 2025Collaborative Inference and Learning between Edge SLMs and Cloud LLMs: A Survey of Algorithms, Execution, and Open Challenges
  5. 2026SPADE: Speculative Decoding for Precise and Low Cost Distributed Edge Cloud Inference
  6. 2026Efficient and Privacy Aware Edge Cloud Collaborative Inference for Large Language Models
  7. 2026Delay-Adaptive Speculation Control for Low-Latency Edge-Cloud LLM Inference
  8. 2026Speculation at a Distance: Where Edge-Cloud Speculative Decoding Actually Pays Off