mlsanityData to 14 Oct 2026

Pallas: A Proactive KV Cache Migration Framework for LLM Inference in AI-RAN

arXiv 2608.16477 ↗First posted 17 Aug 2026cs.LGcited by 0

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

AI-RAN brings large language model (LLM) serving close to mobile users, but cellular handover can separate an active request from its inference state: the user attaches to a target base station (gNB) while the large and growing key-value (KV) cache remains at the source. Retaining inference at the source preserves service continuity but persistently increases inter-token latency (ITL), whereas recovering the state at the target restores serving locality but requires KV-cache transfer, recomputation, or a combination of both only after handover, directly prolonging service interruption time (SIT). This work presents Pallas, a textitproactive KV-cache migration framework that prepares the inference state at the predicted target before handover, in parallel with ongoing source-side inference and token delivery. At the preparation trigger, Pallas partitions the token sequence into a stable historical prefix and an evolving suffix. The target reconstructs the prefix through local prefill, while the source streams the KV blocks generated for the suffix. At handover, the target assembles both portions into an up-to-date KV cache and resumes decoding locally, leaving only unfinished preparation to contribute to SIT. An online scheduler selects the textitprefetching window, which determines how early preparation begins before handover, based on mobility predictions and runtime telemetry. Across three LLMs and 100--500~Mbps inter-gNB links, our vLLM-based prototype reduces average SIT by factors of 2.28--89.68 over target-side recovery approaches and lowers average ITL by 16.0%--50.0% compared with source-side forwarding.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 14 of its 29 references that are in the map. First posted on arXiv 17 Aug 2026.

Builds on

14 of 29 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Serving11 Oct
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttention12 Sep

Other references · 12

  1. 2025Qwen3 Technical Reportcited by 8,820
  2. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 637
  3. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Servingcited by 617
  4. 2023SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefillscited by 193
  5. 2024Llumnix: Dynamic Scheduling for Large Language Model Servingcited by 174
  6. 2024ServerlessLLM: Low-Latency Serverless Inference for Large Language Modelscited by 80
  7. 2022Understanding O-RAN: Architecture, Interfaces, Algorithms, Security, and Research Challengescited by 55
  8. 2025ServeGen: Workload Characterization and Generation of Large Language Model Serving in Productioncited by 29
  9. 2025Prism: Cost-Efficient Multi-LLM Serving via GPU Memory Ballooningcited by 24
  10. 2026Low-Latency Edge LLM Handover via Joint KV Cache Transfer and Token Prefillcited by 4
  11. 2021Realtime Mobile Bandwidth and Handoff Predictions in 4G/5G Networkscited by 3
  12. 2022Follow-Me Cloud: When Cloud Services Follow Mobile Userscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025KVShare: An LLM Service System with Efficient and Effective Multi-Tenant KV Cache Reuse
  2. 2026The KV Cache Working Set: Online Capacity Planning for LLM Inference Systems
  3. 2025Apt-Serve: Adaptive Request Scheduling on Hybrid Cache for Scalable LLM Inference Serving
  4. 2026SmartGen: Seamless Disaggregated LLM Inference with Selective KV Cache Transfer
  5. 2026Comparative Characterization of KV Cache Management Strategies for LLM Inference
  6. 2026KV Cache Optimization Strategies for Scalable and Efficient LLM Inference
  7. 2026SparKV: Overhead-Aware KV Cache Loading for Efficient On-Device LLM Inference
  8. 2026Dynamic Flow, Static Graph: KV Cache Reuse for Efficient LLM Serving on Mobile NPUs