# DisCEdge: Distributed Context Management for Large Language Models at the Edge

arXiv: https://arxiv.org/abs/2511.22599 · HTML: https://mlsanity.com/paper/2511.22599/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 43 references that are in the map.
- First posted on arXiv 27 Nov 2025.

## Builds on (most cited first, 20 of 20)

- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [π0: A Vision-Language-Action Flow Model for General Robot Control](https://mlsanity.com/paper/2410.24164/) (2024)
- [SGLang: Efficient Execution of Structured Language Model Programs](https://mlsanity.com/paper/2312.07104/) (2023)
- [TinyLlama: An Open-Source Small Language Model](https://mlsanity.com/paper/2401.02385/) (2024)
- [Mobile Edge Intelligence for Large Language Models: A Contemporary Survey](https://mlsanity.com/paper/2407.18921/) (2024)
- [A Review on Edge Large Language Models: Design, Execution, and Applications](https://mlsanity.com/paper/2410.11845/) (2024)
- [Token Pruning in Multimodal Large Language Models: Are We Solving the Right Problem?](https://mlsanity.com/paper/2502.11501/) (2025)
- [Recursively Summarizing Enables Long-Term Dialogue Memory in Large Language Models](https://mlsanity.com/paper/2308.15022/) (2023)
- [Stateful Large Language Model Serving with Pensieve](https://mlsanity.com/paper/2312.05516/) (2023)
- [Extending Context Window of Large Language Models via Semantic Compression](https://mlsanity.com/paper/2312.09571/) (2023)
- [Reducing Distraction in Long-Context Language Models by Focused Learning](https://mlsanity.com/paper/2411.05928/) (2024)
- [Leveraging Cloud Computing to Make Autonomous Vehicles Safer](https://mlsanity.com/paper/2308.03204/) (2023)
- [Small Models, Big Tasks: An Exploratory Empirical Study on Small Language Models for Function Calling](https://mlsanity.com/paper/2504.19277/) (2025)
- [FogROS2: An Adaptive Platform for Cloud and Fog Robotics Using ROS 2](https://mlsanity.com/paper/2205.09778/) (2022)
- [FogROS2-PLR: Probabilistic Latency-Reliability For Cloud Robotics](https://mlsanity.com/paper/2410.05562/) (2024)
- [FlexQuant: Elastic Quantization Framework for Locally Hosted LLM on Edge Devices](https://mlsanity.com/paper/2501.07139/) (2025)
- [Managing Data Replication and Distribution in the Fog with FReD](https://mlsanity.com/paper/2303.05256/) (2023)
- [Investigating Text Shortening Strategy in BERT: Truncation vs Summarization](https://mlsanity.com/paper/2403.12799/) (2024)
- [GeoFaaS: An Edge-to-Cloud FaaS Platform](https://mlsanity.com/paper/2405.14413/) (2024)

## Abstract (arXiv)

Deploying Large Language Model (LLM) services at the edge benefits latency-sensitive and privacy-aware applications. However, the stateless nature of LLMs makes managing user context (e.g., sessions, preferences) across geo-distributed edge nodes challenging. Existing solutions, such as client-side context storage, introduce network latency and bandwidth overhead, undermining edge deployment advantages. We propose DisCEdge, a distributed context management system that stores and replicates user context in tokenized form across edge nodes. By maintaining context as token sequences, our system avoids redundant computation and enables efficient data replication. We evaluate an open-source prototype in a realistic edge environment. DisCEdge improves median response times by up to 14.46% and lowers median inter-node synchronization overhead by up to 15% compared to a raw-text-based system. It also reduces client request sizes by a median of 90% compared to client-side context management, while guaranteeing data consistency.