# Serving Long-Context LLMs at the Mobile Edge: Test-Time Reinforcement Learning-based Model Caching and Inference Offloading

arXiv: https://arxiv.org/abs/2501.14205 · HTML: https://mlsanity.com/paper/2501.14205/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jan 2025.

## Built on by (most cited first, 10 of 10)

- [A Survey of Context Engineering for Large Language Models](https://mlsanity.com/paper/2507.13334/) (2025)
- [DeepSeek-Inspired Exploration of RL-based LLMs and Synergy with Wireless Networks: A Survey](https://mlsanity.com/paper/2503.09956/) (2025)
- [OrderMoE: An expert similarity driven distributed edge MoE inference](https://mlsanity.com/paper/2607.17154/) (2026)
- [Federated Attention: A Distributed Paradigm for Collaborative LLM Inference over Edge Networks](https://mlsanity.com/paper/2511.02647/) (2025)
- [Multi-Turn Reasoning LLMs for Task Offloading in Mobile Edge Computing](https://mlsanity.com/paper/2604.07148/) (2026)
- [CR^2: Cost-Aware Risk-Controlled Routing for Wireless Device-Edge LLM Inference](https://mlsanity.com/paper/2605.12001/) (2026)
- [The Price of Anarchy in Disaggregated Inference](https://mlsanity.com/paper/2606.17081/) (2026)
- [LMEdge: QoS-Aware LLM Inference Orchestration on Edge Clusters](https://mlsanity.com/paper/2607.17175/) (2026)
- [Accuracy-Delay Trade-Off in LLM Offloading via Token-Level Uncertainty](https://mlsanity.com/paper/2602.07958/) (2026)
- [POLAR: Online Learning for LoRA Adapter Caching and Routing in Edge LLM Serving](https://mlsanity.com/paper/2604.16583/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) can perform zero-shot learning on unseen tasks and few-shot learning on complex reasoning tasks. However, resource-limited mobile edge networks struggle to support long-context LLM serving for LLM agents during multi-round interactions with users. Unlike stateless computation offloading and static service offloading in edge computing, optimizing LLM serving at edge servers is challenging because LLMs continuously learn from context which raises accuracy, latency, and resource consumption dynamics. In this paper, we propose a joint model caching and inference offloading framework that utilizes test-time deep reinforcement learning (T2DRL) to optimize deployment and execution strategies for long-context LLM serving. In this framework, we analyze the performance convergence and design an optimization problem considering the utilization of context windows in LLMs. Furthermore, the T2DRL algorithm can learn in both the training phase and the testing phase to proactively manage cached models and service requests and adapt to context changes and usage patterns during execution. To further enhance resource allocation efficiency, we propose a double Dutch auction (DDA) mechanism, which dynamically matches supply and demand while maximizing social welfare. Finally, experimental results demonstrate that the T2DRL algorithm can reduce system costs by at least 30% compared to baselines while guaranteeing the performance of LLM agents in real-world perception and reasoning tasks.