# Fast Collaborative Inference via Distributed Speculative Decoding

arXiv: https://arxiv.org/abs/2512.16273 · HTML: https://mlsanity.com/paper/2512.16273/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Dec 2025.

## Built on by (most cited first, 4 of 4)

- [Speculation at a Distance: Where Edge-Cloud Speculative Decoding Actually Pays Off](https://mlsanity.com/paper/2606.25091/) (2026)
- [SpecFed: Accelerating Federated LLM Inference with Speculative Decoding and Compressed Transmission](https://mlsanity.com/paper/2604.25777/) (2026)
- [Multi-SPIN: Multi-Access Speculative Inference for Cooperative Token Generation at the Edge](https://mlsanity.com/paper/2606.04581/) (2026)
- [AsymSpec: Efficient Cloud-Edge Speculative Decoding over Asymmetric Networks](https://mlsanity.com/paper/2608.04974/) (2026)

## Abstract (arXiv)

Speculative decoding accelerates large language model (LLM) inference by allowing a small draft model to predict multiple future tokens for verification by a larger target model. In AI-native radio access networks (AI-RAN), this enables device-edge collaborative inference but introduces significant uplink overhead, as existing distributed speculative decoding schemes transmit full vocabulary logits at every step. We propose a sparsify-then-sample strategy, Truncated Sparse Logits Transmission (TSLT), which transmits only the logits and indices of a truncated candidate set. We provide theoretical guarantees showing that the acceptance rate is preserved under TSLT. TSLT is further extended to multi-candidate case, where multiple draft candidates per step increase acceptance probability. Experiments show that TSLT significantly reduces uplink communication while maintaining end-to-end inference latency and model quality, demonstrating its effectiveness for scalable, communication-efficient distributed LLM inference in future AI-RAN systems.