# DeServe: Towards Affordable Offline LLM Inference via Decentralization

arXiv: https://arxiv.org/abs/2501.14784 · HTML: https://mlsanity.com/paper/2501.14784/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 43 references that are in the map.
- First posted on arXiv 4 Jan 2025.

## Builds on (most cited first, 21 of 21)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism](https://mlsanity.com/paper/1909.08053/) (2019)
- [Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference](https://mlsanity.com/paper/2403.04132/) (2024)
- [Generating Sequences With Recurrent Neural Networks](https://mlsanity.com/paper/1308.0850/) (2013)
- [Efficiently Scaling Transformer Inference](https://mlsanity.com/paper/2211.05102/) (2022)
- [Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve](https://mlsanity.com/paper/2403.02310/) (2024)
- [Mesh-TensorFlow: Deep Learning for Supercomputers](https://mlsanity.com/paper/1811.02084/) (2018)
- [Inference without Interference: Disaggregate LLM Inference for Mixed Downstream Workloads](https://mlsanity.com/paper/2401.11181/) (2024)
- [Petals: Collaborative Inference and Fine-tuning of Large Models](https://mlsanity.com/paper/2209.01188/) (2022)
- [TeraPipe: Token-Level Pipeline Parallelism for Training Large-Scale Language Models](https://mlsanity.com/paper/2102.07988/) (2021)
- [Helix: Serving Large Language Models over Heterogeneous GPUs and Network via Max-Flow](https://mlsanity.com/paper/2406.01566/) (2024)
- [zkLLM: Zero Knowledge Proofs for Large Language Models](https://mlsanity.com/paper/2404.16109/) (2024)
- [LLM Inference Serving: Survey of Recent Advances and Opportunities](https://mlsanity.com/paper/2407.12391/) (2024)
- [PerLLM: Personalized Inference Scheduling with Edge-Cloud Collaboration for Diverse LLM Services](https://mlsanity.com/paper/2405.14636/) (2024)
- [opML: Optimistic Machine Learning on Blockchain](https://mlsanity.com/paper/2401.17555/) (2024)
- [FusionAI: Decentralized Training and Deploying LLMs with Massive Consumer-Level GPUs](https://mlsanity.com/paper/2309.01172/) (2023)
- [Slice-Level Scheduling for High Throughput and Load Balanced LLM Serving](https://mlsanity.com/paper/2406.13511/) (2024)
- [FusionLLM: A Decentralized LLM Training System on Geo-distributed GPUs with Adaptive Compression](https://mlsanity.com/paper/2410.12707/) (2024)
- [Agatha: Smart Contract for DNN Computation](https://mlsanity.com/paper/2105.04919/) (2021)
- [Decentralized AI: Permissionless LLM Inference on POKT Network](https://mlsanity.com/paper/2405.20450/) (2024)

## Built on by (most cited first, 4 of 4)

- [Parallax: Efficient LLM Inference Service over Decentralized Environment](https://mlsanity.com/paper/2509.26182/) (2025)
- [Learning Decentralized LLM Collaboration with Multi-Agent Actor Critic](https://mlsanity.com/paper/2601.21972/) (2026)
- [Trust-Aware Routing for Distributed Generative AI Inference at the Edge](https://mlsanity.com/paper/2603.28622/) (2026)
- [Lodestar: An Online-Learning LLM Inference Router](https://mlsanity.com/paper/2606.00946/) (2026)

## Abstract (arXiv)

The rapid growth of generative AI and its integration into everyday workflows have significantly increased the demand for large language model (LLM) inference services. While proprietary models remain popular, recent advancements in open-source LLMs have positioned them as strong contenders. However, deploying these models is often constrained by the high costs and limited availability of GPU resources. In response, this paper presents the design of a decentralized offline serving system for LLM inference. Utilizing idle GPU resources, our proposed system, DeServe, decentralizes access to LLMs at a lower cost. DeServe specifically addresses key challenges in optimizing serving throughput in high-latency network environments. Experiments demonstrate that DeServe achieves a 6.7x-12.6x improvement in throughput over existing serving system baselines in such conditions.