mlsanityData to 7 Oct 2026

DeServe: Towards Affordable Offline LLM Inference via Decentralization

arXiv 2501.14784 ↗First posted 4 Jan 2025cs.DC cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

The rapid growth of generative AI and its integration into everyday workflows have significantly increased the demand for large language model (LLM) inference services. While proprietary models remain popular, recent advancements in open-source LLMs have positioned them as strong contenders. However, deploying these models is often constrained by the high costs and limited availability of GPU resources. In response, this paper presents the design of a decentralized offline serving system for LLM inference. Utilizing idle GPU resources, our proposed system, DeServe, decentralizes access to LLMs at a lower cost. DeServe specifically addresses key challenges in optimizing serving throughput in high-latency network environments. Experiments demonstrate that DeServe achieves a 6.7x-12.6x improvement in throughput over existing serving system baselines in such conditions.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 21 of its 43 references that are in the map. First posted on arXiv 4 Jan 2025.

Builds on

21 of 43 references.

References · 21

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  4. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,151
  5. 2013Generating Sequences With Recurrent Neural Networkscited by 549
  6. 2022Efficiently Scaling Transformer Inferencecited by 529
  7. 2024Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Servecited by 507
  8. 2018Mesh-TensorFlow: Deep Learning for Supercomputerscited by 168
  9. 2024Inference without Interference: Disaggregate LLM Inference for Mixed Downstream Workloadscited by 120
  10. 2022Petals: Collaborative Inference and Fine-tuning of Large Modelscited by 84
  11. 2021TeraPipe: Token-Level Pipeline Parallelism for Training Large-Scale Language Modelscited by 74
  12. 2024Helix: Serving Large Language Models over Heterogeneous GPUs and Network via Max-Flowcited by 54
9 more
  1. 2024zkLLM: Zero Knowledge Proofs for Large Language Modelscited by 51
  2. 2024LLM Inference Serving: Survey of Recent Advances and Opportunitiescited by 46
  3. 2024PerLLM: Personalized Inference Scheduling with Edge-Cloud Collaboration for Diverse LLM Servicescited by 23
  4. 2023FusionAI: Decentralized Training and Deploying LLMs with Massive Consumer-Level GPUscited by 20
  5. 2024opML: Optimistic Machine Learning on Blockchaincited by 20
  6. 2024Slice-Level Scheduling for High Throughput and Load Balanced LLM Servingcited by 13
  7. 2024FusionLLM: A Decentralized LLM Training System on Geo-distributed GPUs with Adaptive Compressioncited by 7
  8. 2021Agatha: Smart Contract for DNN Computationcited by 5
  9. 2024Decentralized AI: Permissionless LLM Inference on POKT Networkcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2025Parallax: Efficient LLM Inference Service over Decentralized Environmentcited by 12
  2. 2026Learning Decentralized LLM Collaboration with Multi-Agent Actor Criticcited by 7
  3. 2026Trust-Aware Routing for Distributed Generative AI Inference at the Edgecited by 0
  4. 2026Lodestar: An Online-Learning LLM Inference Routercited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Taming the Titans: A Survey of Efficient LLM Inference Serving
  2. 2024Offline Energy-Optimal LLM Serving: Workload-Based Energy Models for LLM Inference on Heterogeneous Systems
  3. 2024Demystifying AI Platform Design for Distributed Inference of Next-Generation LLM models
  4. 2023Splitwise: Efficient generative LLM inference using phase splitting
  5. 2025Glinthawk: A Two-Tiered Architecture for Offline LLM Inference
  6. 2025MoLink: Distributed and Efficient Serving Framework for Large Models
  7. 2024NEO: Saving GPU Memory Crisis with CPU Offloading for Online LLM Inference
  8. 2026Characterizing Parallelism Strategies in LLM Inference: Fundamental Compute-Communication Trade-offs