mlsanityData to 7 Oct 2026

CIDERS: Cloud-Edge LLM Collaborative Learning via Accelerating Personalized Bilevel Optimization

arXiv 2609.15664 ↗First posted 14 Sep 2026cs.DC cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Amid the rapid advancement of physical-world intelligence, cloud-edge collaborative large language models (LLMs) have emerged as a promising roadmap for practical LLM deployment. However, existing cloud-edge paradigms struggle to balance global consensus with local personalization, which fails to satisfy the need for a unified knowledge foundation on the cloud and domain-specific adaptation at the edge. To address this, we introduce, for the first time, a personalized bilevel optimization framework that formalizes cloud-edge LLM collaboration as a dual structure: the upper level optimizes edge-side personalization, while the lower level governs cloud-side knowledge transfer, reaching cloud-edge evolving in coordination. We then propose CIDERS, an efficient solver that decomposes the model into a learnable backbone and a messenger. While the cloud performs knowledge transfer to the learnable backbone, the key lies in embedding global trajectories into each local personalization step via consensus-variate correction to reconcile personalization with consensus. We provide a comprehensive theoretical analysis, including a geometric characterization of the local trajectory and a full convergence guarantee, revealing an explicit trade-off structure between personalization and global convergence. Extensive experiments demonstrate that CIDERS consistently outperforms competitive baselines on the compressed edge path, with 3.1x and 1.7x gains on mathematical reasoning and code generation, respectively, and a 10% relative gain on instruction metrics. Mechanism experiments attribute these gains to early consensus-corrected coordination and task-aware distillation. Overall, CIDERS offers a viable path toward consensus-guided continuous personalization in cloud-edge LLM systems.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 52 of its 60 references that are in the map. First posted on arXiv 14 Sep 2026.

Builds on

52 of 60 references.

References · 52

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  4. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  5. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  6. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  7. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  8. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  9. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  10. 2019Advances and Open Problems in Federated Learningcited by 3,021
  11. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  12. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
38 more
  1. 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,414
  2. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  3. 2019TinyBERT: Distilling BERT for Natural Language Understandingcited by 1,153
  4. 2016Optimization Methods for Large-Scale Machine Learningcited by 1,140
  5. 2020Adaptive Federated Optimizationcited by 1,036
  6. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  7. 2021Model-Contrastive Federated Learningcited by 762
  8. 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
  9. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 665
  10. 2020Personalized Federated Learning with Moreau Envelopescited by 612
  11. 2021Federated Learning Based on Dynamic Regularizationcited by 555
  12. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 537
  13. 2019Federated Learning with Personalization Layerscited by 533
  14. 2021Towards Personalized Federated Learningcited by 462
  15. 2019Tighter Theory for Local SGD on Identical and Heterogeneous Datacited by 266
  16. 2018Approximation Methods for Bilevel Programmingcited by 231
  17. 2023Towards Building the Federated GPT: Federated Instruction Tuningcited by 184
  18. 2024Improving LoRA in Privacy-preserving Federated Learningcited by 158
  19. 2023FederatedScope-LLM: A Comprehensive Package for Fine-tuning Large Language Models in Federated Learningcited by 146
  20. 2024FLoRA: Federated Fine-Tuning Large Language Models with Heterogeneous Low-Rank Adaptationscited by 144
  21. 2019Convergence of Edge Computing and Deep Learning: A Comprehensive Surveycited by 120
  22. 2024Heterogeneous LoRA for Federated Fine-tuning of On-Device Foundation Modelscited by 114
  23. 2022Petals: Collaborative Inference and Fine-tuning of Large Modelscited by 84
  24. 2024Selective Aggregation for Low-Rank Adaptation in Federated Learningcited by 80
  25. 2023Offsite-Tuning: Transfer Learning without Full Modelcited by 77
  26. 2024FedBiOT: LLM Local Fine-tuning in Federated Learning without Full Modelcited by 73
  27. 2024SplitLoRA: A Split Parameter-Efficient Fine-Tuning Framework for Large Language Modelscited by 60
  28. 2024Galaxy: A Resource-Efficient Collaborative Edge AI System for In-situ Transformer Inferencecited by 25
  29. 2025A Survey on Federated Fine-tuning of Large Language Modelscited by 21
  30. 2025Jupiter: Fast and Resource-Efficient Collaborative Inference of Generative LLMs on Edge Devicescited by 15
  31. 2025A Survey on Parameter-Efficient Fine-Tuning for Foundation Models in Federated Learningcited by 15
  32. 2025FedALT: Federated Fine-Tuning through Adaptive Local Training with Rest-of-World LoRAcited by 13
  33. 2024SplitLLM: Collaborative Inference of LLMs for Model Placement and Throughput Optimizationcited by 10
  34. 2025SpecEdge: Scalable Edge-Assisted Serving Framework for Interactive LLMscited by 10
  35. 2024DLoRA: Distributed Parameter-Efficient Fine-Tuning Solution for Large Language Modelcited by 8
  36. 2025Model-Distributed Inference for Large Language Models at the Edgecited by 8
  37. 2026SplitCom: Communication-efficient Split Federated Fine-tuning of LLMs via Temporal Compressioncited by 4
  38. 2026PipeSD: An Efficient Cloud-Edge Collaborative Pipeline Inference Framework with Speculative Decodingcited by 4

50 most cited of 52 All 52 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Collaborative Inference and Learning between Edge SLMs and Cloud LLMs: A Survey of Algorithms, Execution, and Open Challenges
  2. 2024CE-CoLLM: Efficient and Adaptive Large Language Models Through Cloud-Edge Collaboration
  3. 2024EdgeShard: Efficient LLM Inference via Collaborative Edge Computing
  4. 2024AdaSwitch: Adaptive Switching between Small and Large Agents for Effective Cloud-Local Collaborative Learning
  5. 2025Large-Small Model Collaboration for Enhancing Edge-Deployed Small Models
  6. 2024Empirical Guidelines for Deploying LLMs onto Resource-constrained Edge Devices
  7. 2026From Cloud to Crowd: Democratizing LLM Service with Decentralized Edge Collaboration for RAG ↗
  8. 2026Token-Level LLM Collaboration via FusionRoute