mlsanityData to 7 Oct 2026

PlanetServe: A Decentralized, Scalable, and Privacy-Preserving Overlay for Democratizing Large Language Model Serving

arXiv 2504.20101 ↗First posted 27 Apr 2025cs.DC cs.AIcited by 1

On the map: Efficient LLM inference and serving → Serving systems and request scheduling

Abstract (arXiv)

While significant progress has been made in research and development on open-source and cost-efficient large-language models (LLMs), serving scalability remains a critical challenge, particularly for small organizations and individuals seeking to deploy and test their LLM innovations. Inspired by peer-to-peer networks that leverage decentralized overlay nodes to increase throughput and availability, we propose GenTorrent, an LLM serving overlay that harnesses computing resources from decentralized contributors. We identify four key research problems inherent to enabling such a decentralized infrastructure: 1) overlay network organization; 2) LLM communication privacy; 3) overlay forwarding for resource efficiency; and 4) verification of serving quality. This work presents the first systematic study of these fundamental problems in the context of decentralized LLM serving. Evaluation results from a prototype implemented on a set of decentralized nodes demonstrate that GenTorrent achieves a latency reduction of over 50% compared to the baseline design without overlay forwarding. Furthermore, the security features introduce minimal overhead to serving latency and throughput. We believe this work pioneers a new direction for democratizing and scaling future AI serving capabilities.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Serving systems and request scheduling, one of 12 directions in Efficient LLM inference and serving. Builds on 23 of its 67 references that are in the map. First posted on arXiv 27 Apr 2025.

Builds on

23 of 67 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024Preble: Efficient Distributed Prompt Scheduling for LLM Serving8 May
  2. 2024LoongServe: Efficiently Serving Long-Context Large Language Models with Elastic Sequence Parallelism15 Apr
  3. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving18 Jan
  4. 2023Fairness in Serving Large Language Models31 Dec
  5. 2023Fast Distributed Inference Serving for Large Language Models10 May

Other references · 18

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  3. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  4. 2018Hierarchical Neural Story Generationcited by 1,182
  5. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
  6. 2021Measuring Coding Challenge Competence With APPScited by 879
  7. 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 536
  8. 2024InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Managementcited by 242
  9. 2023PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPUcited by 210
  10. 2023LooGLE: Can Long-Context Language Models Understand Long Contexts?cited by 200
  11. 2023CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Servingcited by 199
  12. 2023From Words to Watts: Benchmarking the Energy Costs of Large Language Model Inferencecited by 168
6 more
  1. 2024StableToolBench: Towards Stable Large-Scale Benchmarking on Tool Learning of Large Language Modelscited by 129
  2. 2024ServerlessLLM: Low-Latency Serverless Inference for Large Language Modelscited by 78
  3. 2024DroidSpeak: KV Cache Sharing for Cross-LLM Communication and Multi-LLM Servingcited by 30
  4. 2024Do Large Language Models Need a Content Delivery Network?cited by 15
  5. 2018The latest gossip on BFT consensuscited by 4
  6. 2013Hang With Your Buddies to Resist Intersection Attackscited by 3

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Parallax: Efficient LLM Inference Service over Decentralized Environmentcited by 12

Similar, not cited

Close in wording, no citation either way

  1. 2026WWW.Serve: Interconnecting Global LLM Services through Decentralization ↗
  2. 2025A Survey on Collaborating Small and Large Language Models for Performance, Cost-effectiveness, Cloud-edge Privacy, and Trustworthiness
  3. 2026Securing LLMs in the Wild: Privacy and Security Challenges at the Edge
  4. 2025LLM Applications: Current Paradigms and the Next Frontier
  5. 2025Efficient Multi-Model Orchestration for Self-Hosted Large Language Models
  6. 2025JITServe: SLO-aware LLM Serving with Imprecise Request Information
  7. 2026ExaServe: Large-Scale LLM Serving on Exascale HPC Systems
  8. 2025Viability and Performance of a Private LLM Server for SMBs: A Benchmark Analysis of Qwen3-30B on Consumer-Grade Hardware