# Pallas: A Proactive KV Cache Migration Framework for LLM Inference in AI-RAN

arXiv: https://arxiv.org/abs/2608.16477 · HTML: https://mlsanity.com/paper/2608.16477/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving.
- Builds on 14 of its 29 references that are in the map.
- First posted on arXiv 17 Aug 2026.

## Placed in

- Efficient LLM inference and serving / [KV cache optimization](https://mlsanity.com/inference/kv-cache-optimization/)

## Builds on (most cited first, 14 of 14)

- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache](https://mlsanity.com/paper/2402.02750/) (2024)
- [DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving](https://mlsanity.com/paper/2401.09670/) (2024)
- [CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Serving](https://mlsanity.com/paper/2310.07240/) (2023)
- [SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefills](https://mlsanity.com/paper/2308.16369/) (2023)
- [Llumnix: Dynamic Scheduling for Large Language Model Serving](https://mlsanity.com/paper/2406.03243/) (2024)
- [ServerlessLLM: Low-Latency Serverless Inference for Large Language Models](https://mlsanity.com/paper/2401.14351/) (2024)
- [Understanding O-RAN: Architecture, Interfaces, Algorithms, Security, and Research Challenges](https://mlsanity.com/paper/2202.01032/) (2022)
- [ServeGen: Workload Characterization and Generation of Large Language Model Serving in Production](https://mlsanity.com/paper/2505.09999/) (2025)
- [Prism: Cost-Efficient Multi-LLM Serving via GPU Memory Ballooning](https://mlsanity.com/paper/2505.04021/) (2025)
- [Low-Latency Edge LLM Handover via Joint KV Cache Transfer and Token Prefill](https://mlsanity.com/paper/2603.28018/) (2026)
- [Realtime Mobile Bandwidth and Handoff Predictions in 4G/5G Networks](https://mlsanity.com/paper/2104.12959/) (2021)
- [Follow-Me Cloud: When Cloud Services Follow Mobile Users](https://mlsanity.com/paper/2201.05068/) (2022)

## Abstract (arXiv)

AI-RAN brings large language model (LLM) serving close to mobile users, but cellular handover can separate an active request from its inference state: the user attaches to a target base station (gNB) while the large and growing key-value (KV) cache remains at the source. Retaining inference at the source preserves service continuity but persistently increases inter-token latency (ITL), whereas recovering the state at the target restores serving locality but requires KV-cache transfer, recomputation, or a combination of both only after handover, directly prolonging service interruption time (SIT). This work presents Pallas, a textitproactive KV-cache migration framework that prepares the inference state at the predicted target before handover, in parallel with ongoing source-side inference and token delivery. At the preparation trigger, Pallas partitions the token sequence into a stable historical prefix and an evolving suffix. The target reconstructs the prefix through local prefill, while the source streams the KV blocks generated for the suffix. At handover, the target assembles both portions into an up-to-date KV cache and resumes decoding locally, leaving only unfinished preparation to contribute to SIT. An online scheduler selects the textitprefetching window, which determines how early preparation begins before handover, based on mobility predictions and runtime telemetry. Across three LLMs and 100--500~Mbps inter-gNB links, our vLLM-based prototype reduces average SIT by factors of 2.28--89.68 over target-side recovery approaches and lowers average ITL by 16.0%--50.0% compared with source-side forwarding.