# PerLLM: Personalized Inference Scheduling with Edge-Cloud Collaboration for Diverse LLM Services

arXiv: https://arxiv.org/abs/2405.14636 · HTML: https://mlsanity.com/paper/2405.14636/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 May 2024.

## Built on by (most cited first, 23 of 23)

- [On-Device Language Models: A Comprehensive Review](https://mlsanity.com/paper/2409.00088/) (2024)
- [LLM Inference Serving: Survey of Recent Advances and Opportunities](https://mlsanity.com/paper/2407.12391/) (2024)
- [Minions: Cost-efficient Collaboration Between On-device and Cloud Language Models](https://mlsanity.com/paper/2502.15964/) (2025)
- [Taming the Titans: A Survey of Efficient LLM Inference Serving](https://mlsanity.com/paper/2504.19720/) (2025)
- [Device-Cloud Collaborative LLM Inference with Multi-Modal, Multi-Task, Multi-Turn Conversations](https://mlsanity.com/paper/2502.11007/) (2025)
- [When IoT Meet LLMs: Applications and Challenges](https://mlsanity.com/paper/2411.17722/) (2024)
- [Distributed LLMs and Multimodal Large Language Models: A Survey on Advances, Challenges, and Future Directions](https://mlsanity.com/paper/2503.16585/) (2025)
- [Adaptive Guidance Semantically Enhanced via Multimodal LLM for Edge-Cloud Object Detection](https://mlsanity.com/paper/2509.19875/) (2025)
- [Task-Oriented Semantic Communication in Large Multimodal Models-based Vehicle Networks](https://mlsanity.com/paper/2505.02413/) (2025)
- [DeServe: Towards Affordable Offline LLM Inference via Decentralization](https://mlsanity.com/paper/2501.14784/) (2025)
- [Semi-Clairvoyant Scheduling of Speculative Decoding Requests to Minimize LLM Inference Latency](https://mlsanity.com/paper/2505.17074/) (2025)
- [SAEC: Scene-Aware Enhanced Edge-Cloud Collaborative Industrial Vision Inspection with Multimodal LLM](https://mlsanity.com/paper/2509.17136/) (2025)
- [DiSCo: Device-Server Collaborative LLM-Based Text Streaming Services](https://mlsanity.com/paper/2502.11417/) (2025)
- [Recursive Offloading for LLM Serving in Multi-tier Networks](https://mlsanity.com/paper/2505.16502/) (2025)
- [From Large AI Models to Agentic AI: A Tutorial on Future Intelligent Communications](https://mlsanity.com/paper/2505.22311/) (2025)
- [Efficient Onboard Vision-Language Inference in UAV-Enabled Low-Altitude Economy Networks via LLM-Enhanced Optimization](https://mlsanity.com/paper/2510.10028/) (2025)
- [Adapter-Augmented Bandits for Online Multi-Constrained Multi-Modal Inference Scheduling](https://mlsanity.com/paper/2603.06403/) (2026)
- [MARLIN: Multi-Agent Game-Theoretic Reinforcement Learning for Sustainable LLM Inference in Cloud Datacenters](https://mlsanity.com/paper/2605.13496/) (2026)
- [Pro-Router: Token-Aware Progressive Model Routing with Adaptive Edge-Cloud Collaboration for Efficient Multimodal LLM Inference](https://mlsanity.com/paper/2608.28726/) (2026)
- [Multi-Agentic AI for Fairness-Aware and Accelerated Multi-modal Large Model Inference in Real-world Mobile Edge Networks](https://mlsanity.com/paper/2602.07215/) (2026)
- [CoVStream: Edge-Cloud Collaboration for Understanding of Long Video Streams](https://mlsanity.com/paper/2606.22804/) (2026)
- [MAPS: Memory-Aware Predictive Scheduling Framework for Large Language Model Serving](https://mlsanity.com/paper/2609.15359/) (2026)
- [Sample Count Is Not Enough: Candidate-Generation Strategy Shapes the Energy and Performance of LLM Test-Time Scaling](https://mlsanity.com/paper/2609.19499/) (2026)

## Abstract (arXiv)

With the rapid growth in the number of large language model (LLM) users, it is difficult for bandwidth-constrained cloud servers to simultaneously process massive LLM services in real-time. Recently, edge-cloud infrastructures have been used to improve the processing efficiency of large-scale LLM services. However, the diversity of task requirements and the dynamics of resources pose great challenges to inference scheduling, leading to the wastage of many resources. In this paper, we present PerLLM, a personalized inference scheduling framework with edge-cloud collaboration designed for diverse LLM services. For the complexity of multiple constraints and the decision-making process of edge-cloud collaboration, we integrate the upper confidence bound algorithm based on the constraint satisfaction mechanism in PerLLM. For diverse LLM services, PerLLM can optimize service scheduling and resource allocation solutions within the edge-cloud infrastructure to meet processing time requirements while minimizing energy costs. Experimental results from different model deployments show that PerLLM can effectively meet the processing time requirements of personalized services. Compared to other methods, PerLLM achieves 2.2x, 2.1x, and 1.6x throughput and reduces the energy cost by more than 50%.