Experience-Driven Dynamic Exits for LLMs with Reinforcement Learning
Not on the map. Why?
Abstract (arXiv)
Large Language Models suffer from slow autoregressive inference. While self-speculative decoding accelerates this process, its efficiency is hampered by static configurations like fixed exit layers and speculation lengths. We reframe this optimization as a Markov Decision Process and propose LEDE, a framework that uses offline reinforcement learning. LEDE learns a policy to dynamically select the optimal exit layer and speculation length based on the local context of the generated sequence at each step, balancing computational cost and draft quality. Comprehensive evaluations on Llama-2 and Llama-3 models show LEDE achieves up to a 2.0×~2.7× speedup over autoregressive decoding and and provides an additional 17% speedup over the static speculative baselines.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 23 of its 32 references that are in the map. First posted on arXiv 2 Jun 2026.
Builds on
23 of 32 references.
References · 23
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2023GPT-4 Technical Reportcited by 18,132
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
- 2016Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyondcited by 809
- 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertaintycited by 564
- 2023DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Modelscited by 367
- 2023SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verificationcited by 359
- 2024LayerSkip: Enabling Early Exit Inference and Self-Speculative Decodingcited by 302
- 2022Confident Adaptive Language Modelingcited by 297
- 2024Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decodingcited by 266
11 more
- 2017Noisy Networks for Explorationcited by 262
- 2023Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Timecited by 247
- 2024Mixture-of-Depths: Dynamically allocating compute in transformer-based language modelscited by 242
- 2022Speculative Decoding: Exploiting Speculative Execution for Accelerating Seq2seq Generationcited by 169
- 2024Not All Layers of LLMs Are Necessary During Inferencecited by 95
- 2020Low-Resource Domain Adaptation for Compositional Task-Oriented Semantic Parsingcited by 72
- 2025DAST: Context-Aware Compression in LLMs via Dynamic Allocation of Soft Tokenscited by 21
- 2024Accelerating Inference in Large Language Models with a Unified Layer Skipping Strategycited by 17
- 2025KNN-SSD: Enabling Dynamic Self-Speculative Decoding via Nearest Neighbor Layer Set Optimizationcited by 10
- 2024Draft on the Fly: Adaptive Self-Speculative Decoding using Cosine Similaritycited by 8
- 2025AdmTree: Compressing Lengthy Context with Adaptive Semantic Treescited by 7
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026EfficientRollout: System-Aware Self-Speculative Decoding for RL Rollouts
- 2025SPEC-RL: Accelerating On-Policy Reinforcement Learning with Speculative Rollouts
- 2025ReSpec: Towards Optimizing Speculative Decoding in Reinforcement Learning Systems
- 2026BubbleSpec: Turning Long-Tail Bubbles into Speculative Rollout Drafts for Synchronous Reinforcement Learning
- 2025Beat the long tail: Distribution-Aware Speculative Decoding for RL Training
- 2026When RL Meets Adaptive Speculative Training: A Unified Training-Serving System
- 2026SpecRoll: Fast-Slow Verifier-Feedback Adaptation for Speculative Reinforcement Learning Rollouts
- 2025Trajectory Balance with Asynchrony: Decoupling Exploration and Learning for Fast, Scalable LLM Post-Training