# SLA-Aware Distributed LLM Inference Across Device-RAN-Cloud

arXiv: https://arxiv.org/abs/2602.23722 · HTML: https://mlsanity.com/paper/2602.23722/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 15 references that are in the map.
- First posted on arXiv 27 Feb 2026.

## Builds on (most cited first, 8 of 8)

- [GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers](https://mlsanity.com/paper/2210.17323/) (2022)
- [LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale](https://mlsanity.com/paper/2208.07339/) (2022)
- [AI-RAN: Transforming RAN with AI-driven Computing Infrastructure](https://mlsanity.com/paper/2501.09007/) (2025)
- [Empowering the 6G Cellular Architecture with Open RAN](https://mlsanity.com/paper/2312.02746/) (2023)
- [Beyond Connectivity: An Open Architecture for AI-RAN Convergence in 6G](https://mlsanity.com/paper/2507.06911/) (2025)
- [AutoRAN: Automated and Zero-Touch Open RAN Systems](https://mlsanity.com/paper/2504.11233/) (2025)
- [CoFormer: Collaborating with Heterogeneous Edge Devices for Scalable Transformer Inference](https://mlsanity.com/paper/2508.20375/) (2025)
- [Adaptive AI Model Partitioning over 5G Networks](https://mlsanity.com/paper/2509.01906/) (2025)

## Abstract (arXiv)

Embodied AI requires sub-second inference near the Radio Access Network (RAN), but deployments span heterogeneous tiers (on-device, RAN-edge, cloud) and must not disrupt real-time baseband processing. We report measurements from a 5G Standalone (SA) AI-RAN testbed using a fixed baseline policy for repeatability. The setup includes an on-device tier, a three-node RAN-edge cluster co-hosting a containerized 5G RAN, and a cloud tier. We find that on-device execution remains multi-second and fails to meet sub-second budgets. At the RAN edge, SLA feasibility is primarily determined by model variant choice: quantized models concentrate below 0.5,s, while unquantized and some larger quantized models incur deadline misses due to stalls and queuing. In the cloud tier, meeting a 0.5,s deadline is challenging on the measured WAN path (up to 32.9% of requests complete within 0.5,s), but all evaluated variants meet a 1.0,s deadline (100% within 1.0,s). Under saturated downlink traffic and up to N=20 concurrent inference clients, Multi-Instance GPU (MIG) isolation preserves baseband timing-health proxies, supporting safe co-location under fixed partitioning.