mlsanityData to 14 Oct 2026

Synera: Synergistic LLM Serving across Device and Cloud at Scale

arXiv 2511.07423 ↗First posted 17 Oct 2025cs.DC cs.AI cs.LGcited by 0

On the map: Efficient LLM inference and serving → Serving systems and request scheduling

Abstract (arXiv)

Large Language Models (LLMs) are becoming key components in various mobile operating systems, driving smart applications like interactive chatbots and personal assistants. While bringing enhanced intelligence to mobile ends, their deployment suffers from a set of performance challenges, especially the generation quality degradation and prolonged latency. Prior works have mainly relied on solutions of cloud offloading or on-device Small Language Models (SLMs). However, the former is usually limited by the communication bottleneck, and the latter sacrifices generation quality due to resource constraints. To mitigate these limitations, this paper proposes Synera, a device-cloud synergistic LLM serving system that applies an efficient SLM-LLM synergistic mechanism. Through empirical studies on LLM's unique computing characteristics, Synera identifies a set of underexplored optimization opportunities in device-cloud synergistic LLM inference, including offloading decisions, pipeline stalls, and batching bottlenecks. To translate them into enhanced performance, Synera introduces tailored designs of communication-efficient selective offloading, stall-free parallel inference, and scalable cloud batching. Extensive evaluations with real-world testbeds show that Synera enables 1.20-5.47x better generation quality against competitive baselines with on-par latency performance. Compared with existing cloud serving, Synera achieves 8.2-16.5% lower cloud serving cost on various benchmarks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Serving systems and request scheduling, one of 12 directions in Efficient LLM inference and serving. Builds on 26 of its 50 references that are in the map. First posted on arXiv 17 Oct 2025.

Builds on

26 of 50 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve4 Mar

Other references · 25

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  3. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,822
  4. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,317
  5. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  6. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,292
  7. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,121
  8. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 1,006
  9. 2024Yi: Open Foundation Models by 01.AIcited by 719
  10. 2024TinyLlama: An Open-Source Small Language Modelcited by 618
  11. 2023The Falcon Series of Open Language Modelscited by 535
  12. 2023SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verificationcited by 365
13 more
  1. 2025Small Language Models are the Future of Agentic AIcited by 270
  2. 2019Edge AI: On-Demand Accelerating Deep Neural Network Inference via Edge Computingcited by 152
  3. 2020SPINN: Synergistic Progressive Inference of Neural Networks over Device and Cloudcited by 112
  4. 2020CoEdge: Cooperative DNN Inference with Adaptive Workload Partitioning over Heterogeneous Edge Devicescited by 67
  5. 2025SpecEE: Accelerating Large Language Model Inference with Speculative Early Exitingcited by 23
  6. 2023EdgeFM: Leveraging Foundation Model for Open-set Learning on the Edgecited by 21
  7. 2023LUT-NN: Empower Efficient Neural Network Inference with Centroid Learning and Table Lookupcited by 20
  8. 2021AppealNet: An Efficient and Highly-Accurate Edge/Cloud Collaborative Architecture for DNN Inferencecited by 17
  9. 2023Real-time Neural Network Inference on Extremely Weak Devices: Agile Offloading with Explainable AIcited by 17
  10. 2023HeySQuAD: A Spoken Question Answering Datasetcited by 16
  11. 2025Jupiter: Fast and Resource-Efficient Collaborative Inference of Generative LLMs on Edge Devicescited by 15
  12. 2024Asteroid: Resource-Efficient Hybrid Pipeline Parallelism for Collaborative DNN Training on Heterogeneous Edge Devicescited by 13
  13. 2025SensorQA: A Question Answering Benchmark for Daily-Life Monitoringcited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Device-Cloud Collaborative LLM Inference with Multi-Modal, Multi-Task, Multi-Turn Conversations
  2. 2025Collaborative Inference and Learning between Edge SLMs and Cloud LLMs: A Survey of Algorithms, Execution, and Open Challenges
  3. 2025TAPAS: Thermal- and Power-Aware Scheduling for LLM Inference in Cloud Platforms
  4. 2026E2LLM: Towards Efficient LLM Serving in Heterogeneous Edge/Fog Environments
  5. 2026When NPUs Are Not Always Faster: A Stage-Level Analysis of Mobile LLM Inference
  6. 2025Recursive Offloading for LLM Serving in Multi-tier Networks
  7. 2026SARA: SLO-Aware Resource Allocation for Disaggregated Agentic LLM Services
  8. 2026Federated Inference for Heterogeneous LLM Communication and Collaboration