# CE-CoLLM: Efficient and Adaptive Large Language Models Through Cloud-Edge Collaboration

arXiv: https://arxiv.org/abs/2411.02829 · HTML: https://mlsanity.com/paper/2411.02829/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 55 of its 84 references that are in the map.
- First posted on arXiv 5 Nov 2024.

## Builds on (most cited first, 30 of 55)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions](https://mlsanity.com/paper/1905.10044/) (2019)
- [Mixtral of Experts](https://mlsanity.com/paper/2401.04088/) (2024)
- [Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization](https://mlsanity.com/paper/1808.08745/) (2018)
- [AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration](https://mlsanity.com/paper/2306.00978/) (2023)
- [A Survey on Evaluation of Large Language Models](https://mlsanity.com/paper/2307.03109/) (2023)
- [SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot](https://mlsanity.com/paper/2301.00774/) (2023)
- [Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond](https://mlsanity.com/paper/1602.06023/) (2016)
- [A Simple and Effective Pruning Approach for Large Language Models](https://mlsanity.com/paper/2306.11695/) (2023)
- [A Comprehensive Overview of Large Language Models](https://mlsanity.com/paper/2307.06435/) (2023)
- [BranchyNet: Fast Inference via Early Exiting from Deep Neural Networks](https://mlsanity.com/paper/1709.01686/) (2017)
- [Splitwise: Efficient generative LLM inference using phase splitting](https://mlsanity.com/paper/2311.18677/) (2023)
- [Adaptive Computation Time for Recurrent Neural Networks](https://mlsanity.com/paper/1603.08983/) (2016)
- [QuAC : Question Answering in Context](https://mlsanity.com/paper/1808.07036/) (2018)
- [SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression](https://mlsanity.com/paper/2306.03078/) (2023)
- [DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference](https://mlsanity.com/paper/2004.12993/) (2020)
- [The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits](https://mlsanity.com/paper/2402.17764/) (2024)
- [BERT Loses Patience: Fast and Robust Inference with Early Exit](https://mlsanity.com/paper/2006.04152/) (2020)
- [Challenges and Applications of Large Language Models](https://mlsanity.com/paper/2307.10169/) (2023)
- [DynaBERT: Dynamic BERT with Adaptive Width and Depth](https://mlsanity.com/paper/2004.04037/) (2020)
- [Depth-Adaptive Transformer](https://mlsanity.com/paper/1910.10073/) (2019)

## Built on by (most cited first, 25 of 25)

- [Minions: Cost-efficient Collaboration Between On-device and Cloud Language Models](https://mlsanity.com/paper/2502.15964/) (2025)
- [A Survey on Collaborative Mechanisms Between Large and Small Language Models](https://mlsanity.com/paper/2505.07460/) (2025)
- [Intelligence per Watt: Measuring Intelligence Efficiency of Local AI](https://mlsanity.com/paper/2511.07885/) (2025)
- [Cognitive Edge Computing: A Comprehensive Survey on Optimizing Large Models and AI Agents for Pervasive Deployment](https://mlsanity.com/paper/2501.03265/) (2025)
- [Bridging On-Device and Cloud LLMs for Collaborative Reasoning: A Unified Methodology for Local Routing and Post-Training](https://mlsanity.com/paper/2509.24050/) (2025)
- [PLM: Efficient Peripheral Language Models Hardware-Co-Designed for Ubiquitous Computing](https://mlsanity.com/paper/2503.12167/) (2025)
- [Multi-turn Jailbreaking Attack in Multi-Modal Large Language Models](https://mlsanity.com/paper/2601.05339/) (2026)
- [Creation of AI-driven Smart Spaces for Enhanced Indoor Environments -- A Survey](https://mlsanity.com/paper/2412.14708/) (2024)
- [Collaborative Large Language Model Inference via Resource-Aware Parallel Speculative Decoding](https://mlsanity.com/paper/2511.01695/) (2025)
- [Lever: Speculative LLM Inference on Smartphones](https://mlsanity.com/paper/2605.16786/) (2026)
- [INAR-VL: Input-Aware Routing for Edge-Cloud Vision-Language Inference](https://mlsanity.com/paper/2605.18853/) (2026)
- [Federated Attention: A Distributed Paradigm for Collaborative LLM Inference over Edge Networks](https://mlsanity.com/paper/2511.02647/) (2025)
- [A Structure-Agnostic Co-Tuning Framework for LLMs and SLMs in Cloud-Edge Systems](https://mlsanity.com/paper/2511.11678/) (2025)
- [Jailbreaking Large Vision Language Models in Intelligent Transportation Systems](https://mlsanity.com/paper/2511.13892/) (2025)
- [Cloud-Edge Collaborative Large Models for Robust Photovoltaic Power Forecasting](https://mlsanity.com/paper/2603.22343/) (2026)
- [CR^2: Cost-Aware Risk-Controlled Routing for Wireless Device-Edge LLM Inference](https://mlsanity.com/paper/2605.12001/) (2026)
- [Efficient and Privacy Aware Edge Cloud Collaborative Inference for Large Language Models](https://mlsanity.com/paper/2607.13093/) (2026)
- [QuickGrasp: Responsive Video-Language Querying Service via Accelerated Tokenization and Edge-Augmented Inference](https://mlsanity.com/paper/2603.00126/) (2026)
- [Networking-Aware Energy Efficiency in Agentic AI Inference: A Survey](https://mlsanity.com/paper/2604.07857/) (2026)
- [Multi-Agent Systems: From Classical Paradigms to Large Foundation Model-Enabled Futures](https://mlsanity.com/paper/2604.18133/) (2026)
- [CONCORD: Asynchronous Sparse Aggregation for Device-Cloud RAG under Document Isolation](https://mlsanity.com/paper/2606.15179/) (2026)
- [Enabling Cloud-Level Accuracy in Edge AI through IoT Data Preprocessing](https://mlsanity.com/paper/2606.22496/) (2026)
- [SkillSmith: Enhancing Locally Deployed Agents via Automatic Skill Construction and Evolution](https://mlsanity.com/paper/2608.08037/) (2026)
- [P2Skill: Privacy Preserving Skill Distillation for Cloud-Local LLM Inference Systems](https://mlsanity.com/paper/2608.14094/) (2026)
- [Ask Without Telling: Local SLMs Consult Cloud LLMs Without Revealing Task Intent](https://mlsanity.com/paper/2609.32642/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) exhibit remarkable human-like predictive capabilities. However, it is challenging to deploy LLMs to provide efficient and adaptive inference services at the edge. This paper proposes a novel Cloud-Edge Collaboration framework for LLMs (CE-CoLLM) to tackle these challenges. First, we identify the transmission of LLM contextual data between the cloud and edge as a key performance bottleneck, which introduces substantial communication overhead that dominates overall inference latency and makes na"ive cloud-edge collaboration for LLMs inefficient. Second, we introduce a suite of novel techniques, including a latency-aware early exit mechanism and efficient cloud context management, into CE-CoLLM, which collectively reduce communication overhead and preserve LLM inference accuracy. Third, we design two adaptive inference modes to accommodate diverse edge environments: (1) a low-latency standalone edge inference mode that enables reliable edge-side independent LLM inference even under unstable network conditions, and (2) a high-accuracy cloud-edge collaborative inference mode that adaptively leverages cloud resources to enhance prediction accuracy. Extensive experiments on multiple benchmark datasets demonstrate that CE-CoLLM reduces overall inference time by up to 13.81% and offloads over 84.53% of the computational workload from the cloud to the edge, compared to conventional cloud-based LLM deployment, without sacrificing prediction accuracy. The code is provided on GitHub at https://github.com/mlsysx/CE-CoLLM.