mlsanityData to 14 Oct 2026

SpecFed: Accelerating Federated LLM Inference with Speculative Decoding and Compressed Transmission

arXiv 2604.25777 ↗First posted 28 Apr 2026eess.SP cs.DCcited by 0

On the map: Efficient LLM inference and serving → Speculative decoding

Abstract (arXiv)

Federated inference enhances LLM performance in edge computing through weighted averaging of distributed model predictions. However, autoregressive LLM inference requires frequent full-model forward passes across workers, severely limiting decoding throughput. Distributed deployment further aggravates this due to a communication bottleneck: each worker must transmit full token probability distributions per draft token, dominating end-to-end latency. To address these challenges, we introduce speculative decoding to enable parallel LLM processing and propose a top-K compressed transmission scheme with two server-side reconstruction strategies. We theoretically analyze the robustness of our method in terms of local reconstruction error, aggregation bias, and acceptance-rate bias, and derive corresponding bounds. Experiments demonstrate that our scheme achieves high generation fidelity while significantly reducing communication overhead.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving. Builds on 18 of its 25 references that are in the map. First posted on arXiv 28 Apr 2026.

Builds on

18 of 25 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Judge Decoding: Faster Speculative Sampling Requires Going Beyond Model Alignment31 Jan
  2. 2023Accelerating Large Language Model Decoding with Speculative Sampling2 Feb
  3. 2022Fast Inference from Transformers via Speculative Decoding30 Nov

Other references · 15

  1. 2024The Effect of Sampling Temperature on Problem Solving in Large Language Modelscited by 170
  2. 2025Harnessing Multiple Large Language Models: A Survey on LLM Ensemblecited by 86
  3. 2025Exploring the Impact of Temperature on Large Language Models:Hot or Cold?cited by 40
  4. 2024Breaking the Ceiling of the LLM Community by Treating Token Generation as a Classification for Ensemblingcited by 33
  5. 2024Determine-Then-Ensemble: Necessity of Top-k Union for Large Language Model Ensemblingcited by 28
  6. 2024Uncertainty-Aware Hybrid Inference with On-Device Small and Remote Large Language Modelscited by 19
  7. 2025DSSD: Efficient Edge-Device LLM Deployment and Collaborative Inference via Distributed Split Speculative Decodingcited by 15
  8. 2025Fast Large Language Model Collaborative Decoding via Speculationcited by 12
  9. 2022Over-the-Air Ensemble Inference with Model Privacycited by 10
  10. 2022Decentralized Low-Latency Collaborative Inference via Ensembles on the Edgecited by 10
  11. 2025Quantize-Sample-and-Verify: LLM Acceleration via Adaptive Edge-Cloud Speculative Decodingcited by 4
  12. 2025Communication-Efficient Collaborative LLM Inference via Distributed Speculative Decodingcited by 4
3 more
  1. 2025Efficient LLM Inference over Heterogeneous Edge Networks with Speculative Decodingcited by 4
  2. 2025Fast Collaborative Inference via Distributed Speculative Decodingcited by 4
  3. 2025When to Ensemble: Identifying Token-Level Points for Stable and Fast LLM Ensemblingcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Distributed Generative Inference of LLM at Internet Scales with Multi-Dimensional Communication Optimization
  2. 2026WISP: Waste- and Interference-Suppressed Distributed Speculative LLM Serving at the Edge via Dynamic Drafting and SLO-Aware Batching
  3. 2026Speculation at a Distance: Where Edge-Cloud Speculative Decoding Actually Pays Off
  4. 2026BALANCE: Hybrid Autoregressive-Speculative LLM Inference at the Network Edge
  5. 2025Speculative Decoding in Decentralized LLM Inference: Turning Communication Latency into Computation Throughput
  6. 2026Compiler-Assisted Speculative Sampling for Accelerated LLM Inference on Heterogeneous Edge Devices
  7. 2026A Pipelined Collaborative Speculative Decoding Framework for Efficient Edge-Cloud LLM Inference
  8. 2026UnionSparse: An Index-Efficient Sparsity Framework for Low-Bit Sparse LLM Inference on Edge