mlsanityData to 14 Oct 2026

Recursive Offloading for LLM Serving in Multi-tier Networks

arXiv 2505.16502 ↗First posted 22 May 2025cs.DC cs.NIcited by 2

On the map: Efficient LLM inference and serving → Serving systems and request scheduling

Abstract (arXiv)

Heterogeneous device-edge-cloud computing infrastructures have become widely adopted in telecommunication operators and Wide Area Networks (WANs), offering multi-tier computational support for emerging intelligent services. With the rapid proliferation of Large Language Model (LLM) services, efficiently coordinating inference tasks and reducing communication overhead within these multi-tier network architectures becomes a critical deployment challenge. Existing LLM serving paradigms exhibit significant limitations: on-device deployment supports only lightweight LLMs due to hardware constraints, while cloud-centric deployment suffers from resource congestion and considerable prompt communication overhead caused by frequent service requests during peak periods. Although the model-cascading-based inference strategy adapts better to multi-tier networks, its reliance on fine-grained, manually adjusted thresholds makes it less responsive to dynamic network conditions and varying task complexities. To address these challenges, we propose RecServe, a recursive offloading framework tailored for LLM serving in multi-tier networks. RecServe integrates a task-specific hierarchical confidence evaluation mechanism that guides offloading decisions based on inferred task complexity in progressively scaled LLMs across device, edge, and cloud tiers. To further enable intelligent task routing across tiers, RecServe employs a sliding-window-based dynamic offloading strategy with quantile interpolation, enabling real-time tracking of historical confidence distributions and adaptive offloading threshold adjustments. Experiments on eight datasets demonstrate that RecServe outperforms CasServe in both service quality and communication efficiency, and reduces the communication burden by over 50% compared to centralized cloud-based serving.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Serving systems and request scheduling, one of 12 directions in Efficient LLM inference and serving. Builds on 39 of its 84 references that are in the map. First posted on arXiv 22 May 2025.

Builds on

39 of 84 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving18 Jan

Other references · 38

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2023GPT-4 Technical Reportcited by 18,396
  5. 2022Training language models to follow instructions with human feedbackcited by 15,971
  6. 2024The Llama 3 Herd of Modelscited by 14,715
  7. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  8. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  9. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  10. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  11. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  12. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
26 more
  1. 2025Qwen2.5-VL Technical Reportcited by 5,587
  2. 2024Qwen2.5 Technical Reportcited by 4,942
  3. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  4. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  5. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  6. 2023Qwen Technical Reportcited by 3,344
  7. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  8. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
  9. 2024MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategiescited by 723
  10. 2023A Comprehensive Overview of Large Language Modelscited by 648
  11. 2024MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Casescited by 223
  12. 2023CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Servingcited by 203
  13. 2021Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challengescited by 115
  14. 2020Deep Encoder, Shallow Decoder: Reevaluating Non-autoregressive Machine Translationcited by 105
  15. 2023Pushing Large Language Models to the 6G Edge: Vision, Challenges, and Opportunitiescited by 90
  16. 2024ServerlessLLM: Low-Latency Serverless Inference for Large Language Modelscited by 80
  17. 2021Understanding Softmax Confidence and Uncertaintycited by 63
  18. 2022GEMEL: Model Merging for Memory-Efficient, Real-Time Video Analytics at the Edgecited by 32
  19. 2024Putting GPT-4o to the Sword: A Comprehensive Evaluation of Language, Vision, Speech, and Multimodal Proficiencycited by 31
  20. 2022Mandheling: Mixed-Precision On-Device DNN Training with DSP Offloadingcited by 29
  21. 2024PerLLM: Personalized Inference Scheduling with Edge-Cloud Collaboration for Diverse LLM Servicescited by 23
  22. 2024LLM for Mobile: An Initial Roadmapcited by 11
  23. 2023Agglomerative Federated Learning: Empowering Larger Model Training via End-Edge-Cloud Collaborationcited by 10
  24. 2022Hierarchical Federated Learning with Momentum Acceleration in Multi-Tier Networkscited by 6
  25. 2025DiSCo: Device-Server Collaborative LLM-Based Text Streaming Servicescited by 2
  26. 2025Beyond Model Scale Limits: End-Edge-Cloud Federated Learning with Self-Rectified Knowledge Agglomerationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Accuracy-Delay Trade-Off in LLM Offloading via Token-Level Uncertaintycited by 0
  2. 2026A-MADiff: Attention-Guided Multi-Agent DRL with Diffusion Policies for Memory-Aware Task Orchestration in Mobile AIGC Networkscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Niyama : Breaking the Silos of LLM Inference Serving
  2. 2025Synera: Synergistic LLM Serving across Device and Cloud at Scale
  3. 2025SageServe: Optimizing LLM Serving on Cloud Data Centers with Forecast Aware Auto-Scaling
  4. 2025Collaborative Inference and Learning between Edge SLMs and Cloud LLMs: A Survey of Algorithms, Execution, and Open Challenges
  5. 2025A Predictive and Synergistic Two-Layer Scheduling Framework for LLM Serving
  6. 2026DySCo: Dynamic Sharding for Collaborative Edge-Cloud LLM Inference with Depth-Synchronized Batching
  7. 2025Device-Cloud Collaborative LLM Inference with Multi-Modal, Multi-Task, Multi-Turn Conversations
  8. 2026WANSpec: Leveraging Global Compute Capacity for LLM Inference