# DRLM: Deep Reinforcement Learning-Based LLM Query Orchestration in Edge Environments

arXiv: https://arxiv.org/abs/2609.00442 · HTML: https://mlsanity.com/paper/2609.00442/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 21 references that are in the map.
- First posted on arXiv 31 Aug 2026.

## Builds on (most cited first, 15 of 15)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge](https://mlsanity.com/paper/1811.00937/) (2018)
- [RouteLLM: Learning to Route LLMs with Preference Data](https://mlsanity.com/paper/2406.18665/) (2024)
- [DynamoLLM: Designing LLM Inference Clusters for Performance and Energy Efficiency](https://mlsanity.com/paper/2408.00741/) (2024)
- [ExeGPT: Constraint-Aware Resource Scheduling for LLM Inference](https://mlsanity.com/paper/2404.07947/) (2024)
- [OptLLM: Optimal Assignment of Queries to Large Language Models](https://mlsanity.com/paper/2405.15130/) (2024)
- [Edge-First Language Model Inference: Models, Metrics, and Tradeoffs](https://mlsanity.com/paper/2505.16508/) (2025)
- [Task-Aware LLM Routing with Multi-Level Task-Profile-Guided Data Synthesis for Cold-Start Scenarios](https://mlsanity.com/paper/2604.09377/) (2026)
- [Towards AI-Assisted Sustainable Adaptive Video Streaming Systems: Tutorial and Survey](https://mlsanity.com/paper/2406.02302/) (2024)
- [Robust Batch-Level Query Routing for Large Language Models under Cost and Capacity Constraints](https://mlsanity.com/paper/2603.26796/) (2026)
- [ELLMPEG: An Edge-based Agentic LLM Video Processing Tool](https://mlsanity.com/paper/2602.00028/) (2026)
- [ClusterLess: Deadline-Aware Serverless Workflow Orchestration on Federated Edge Clusters](https://mlsanity.com/paper/2605.04310/) (2026)
- [LMEdge: QoS-Aware LLM Inference Orchestration on Edge Clusters](https://mlsanity.com/paper/2607.17175/) (2026)

## Abstract (arXiv)

Large language model (LLM) services increasingly process heterogeneous queries with diverse latency, accuracy, and resource requirements. While edge deployment reduces response time, the heterogeneity of devices and the diversity of model families, parameter scales, and quantization levels make efficient LLM query orchestration challenging. This paper introduces DRLM, a Deep Reinforcement Learning-based LLM query orchestration framework in edge environments. DRLM integrates two lightweight predictors: (i) a class-conditioned quality estimator that maps queries to semantic categories and infers model performance, and (ii) a feature-driven latency predictor that estimates inference time across model-device configurations. These predictions, combined with system state, feed a factorized Proximal Policy Optimization (PPO) agent that performs state-aware orchestration decisions. To enable data-driven orchestration, we construct a large-scale benchmarking dataset with 223 835 measurements spanning 1258 queries, 6 query classes, 8 model families (32 deployed instances), 5 quantization levels, and heterogeneous edge devices. Evaluation on a 64-node edge cluster and comparison with three baselines and two state-of-the-art methods show that DRLM reduces inference latency by up to 51% and queuing delay by up to 67 %, while incurring at most 8% accuracy loss. It improves latency under increasing workloads up to 61.4%, demonstrating robust and stable orchestration.