LASER: Load-Aware Serving with Early-Exit for Reasoning LLMs at the Edge
On the map: Efficient LLM inference and serving → Serving systems and request scheduling
Abstract (arXiv)
Large reasoning models (LRMs) such as DeepSeek-R1 have achieved strong performance through extended chain-of-thought (CoT) generation. However, deploying them on edge devices raises a conflict between long CoT sequences and constrained resources. Recent confidence-based early exit methods reduce CoT length for individual requests, yet they apply fixed thresholds from a single-request perspective, ignoring multi-request concurrency and load fluctuation in edge serving. To bridge this gap, we propose underlineLoad-underlineAware underlineServing with underlineEarly-exit for underlineReasoning (LASER). LASER couples two complementary designs: (1) a load-aware adaptive exit threshold that adjusts the confidence bar based on real-time system load within an empirically validated robust range, and (2) a difficulty- and load-aware reasoning budget pre-allocation that assigns compute resources by request difficulty and system capacity. We formulate the problem as a joint optimization of reasoning quality and service latency. Experiments on two reasoning models, four benchmarks, and diverse load conditions show that LASER reduces average latency by 17--38% and improves service-level objective (SLO) satisfaction by 3--6% over fixed-threshold baselines, at an average accuracy cost of only 1%.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Serving systems and request scheduling, one of 12 directions in Efficient LLM inference and serving. Builds on 17 of its 24 references that are in the map. First posted on arXiv 30 Jun 2026.
Builds on
17 of 24 references.
Closest ancestors · 4
Same direction, newest first
- 2025SpecEdge: Scalable Edge-Assisted Serving Framework for Interactive LLMs16 May
- 2024Llumnix: Dynamic Scheduling for Large Language Model Serving5 Jun
- 2024Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve4 Mar
- 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving18 Jan
Other references · 13
- 2021Training Verifiers to Solve Math Word Problemscited by 9,743
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
- 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
- 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
- 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
- 2024Qwen2.5-Coder Technical Reportcited by 1,151
- 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMscited by 564
- 2025L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learningcited by 381
- 2024LayerSkip: Enabling Early Exit Inference and Self-Speculative Decodingcited by 302
- 2025Chain of Draft: Thinking Faster by Writing Lesscited by 290
- 2025Training Language Models to Reason Efficientlycited by 242
- 2025Dynamic Early Exit in Reasoning Modelscited by 217
1 more
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025EdgeReasoning: Characterizing Reasoning LLM Deployment on Edge GPUs
- 2026Understanding Inference Scaling for LLMs: Bottlenecks, Trade-offs, and Performance Principles
- 2026Queueing-Aware Optimization of Reasoning Tokens for Accuracy-Latency Trade-offs in LLM Servers
- 2026PASCAL: A Phase-Aware Scheduling Algorithm for Serving Reasoning-based Large Language Models
- 2026Efficient Reasoning on the Edge
- 2026GrowPage: On-Demand KV Budgeting for Efficient LLM Reasoning Serving
- 2026Position: LLM Serving Needs Mathematical Optimization and Algorithmic Foundations, Not Just Heuristics
- 2025Recursive Offloading for LLM Serving in Multi-tier Networks