mlsanityData to 14 Oct 2026

Fast LLM Post-training via Decoupled and Fastest-of-N Speculation

arXiv 2511.16193 ↗First posted 20 Nov 2025cs.DC cs.AIcited by 4

On the map: Efficient LLM inference and serving → Speculative decoding

Abstract (arXiv)

Rollout dominates the training time in large language model (LLM) post-training, where the trained model is used to generate tokens given a batch of prompts. This work, SpecActor, achieves fast rollout with speculative decoding that deploys a fast draft path to accelerate the unparallelizable generation, while the correctness is guaranteed by fast parallel verification of the outputs with the original model. SpecActor addresses two foundational challenges that hinder speculation efficiency: (1) a Decoupled speculation method that overcomes the computation inefficiency issue when executing speculative decoding with relative large per-worker batch size -- a common configuration in training but unfriendly to speculation, and (2) a Fastest-of-N speculation method that selects and combines different draft methods according to the rollout progress to approximate the optimal draft method even when the best one is unknown a priori. Extensive evaluations on production traces show that SpecActor accelerates mean rollout speed by 2.0--2.4x, with up to 2.7x speedup, over common post-training baselines. The results are consistent across both dense and MoE models and across different RL algorithms. Notably, SpecActor is 1.1--2.6x faster compared to vanilla speculative rollout in different traces. The accelerated rollout achieves 1.4--2.3x faster end-to-end training time.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving. Builds on 58 of its 74 references that are in the map. First posted on arXiv 20 Nov 2025.

Builds on

58 of 74 references.

Closest ancestors · 15

Same direction, newest first

  1. 2025ReSpec: Towards Optimizing Speculative Decoding in Reinforcement Learning Systems30 Oct
  2. 2025Scaling Up, Speeding Up: A Benchmark of Speculative Decoding for Efficient LLM Test-Time Scaling30 Aug
  3. 2025SwiftSpec: Ultra-Low Latency LLM Decoding by Scaling Asynchronous Speculative Decoding12 Jun
  4. 2025MoESD: Unveil Speculative Decoding's Potential for Accelerating Sparse MoE26 May
  5. 2025EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test3 Mar
  6. 2024SAM Decoding: Speculative Decoding via Suffix Automaton16 Nov
  7. 2024PEARL: Parallel Speculative Decoding with Adaptive Draft Length13 Aug
  8. 2024EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees24 Jun
  9. 2024TurboSpec: Closed-loop Speculation Control System for Optimizing LLM Serving Goodput20 Jun
  10. 2024Break the Sequential Dependency of LLM Inference Using Lookahead Decoding3 Feb
  11. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty26 Jan
  12. 2024Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding15 Jan
3 more
  1. 2023Online Speculative Decoding11 Oct
  2. 2023Accelerating Large Language Model Decoding with Speculative Sampling2 Feb
  3. 2022Fast Inference from Transformers via Speculative Decoding30 Nov

Other references · 43

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2025Qwen3 Technical Reportcited by 8,820
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  5. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  6. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  7. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  8. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  9. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  10. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  11. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,352
  12. 2023AgentBench: Evaluating LLMs as Agentscited by 1,267
31 more
  1. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
  2. 2025Kimi K2: Open Agentic Intelligencecited by 567
  3. 2024FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precisioncited by 519
  4. 2018Blockwise Parallel Decoding for Deep Autoregressive Modelscited by 323
  5. 2025VAPO: Efficient and Reliable Reinforcement Learning for Advanced Reasoning Taskscited by 285
  6. 2023DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Modelscited by 245
  7. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 205
  8. 2025Multi-SWE-bench: A Multilingual Benchmark for Issue Resolvingcited by 171
  9. 2025Seed1.5-Thinking: Advancing Superb Reasoning Models with Reinforcement Learningcited by 161
  10. 2024LoongServe: Efficiently Serving Long-Context Large Language Models with Elastic Sequence Parallelismcited by 112
  11. 2024Understanding the performance gap between online and offline alignment algorithmscited by 110
  12. 2024NanoFlow: Towards Optimal Large Language Model Serving Throughputcited by 90
  13. 2018Hessian-based Analysis of Large Batch Training and Robustness to Adversariescited by 87
  14. 2024ServerlessLLM: Low-Latency Serverless Inference for Large Language Modelscited by 80
  15. 2025Tapered Off-Policy REINFORCE: Stable and efficient reinforcement learning for LLMscited by 61
  16. 2025StreamRL: Scalable, Heterogeneous, and Elastic RL for LLMs with Disaggregated Stream Generationcited by 61
  17. 2024Optimizing RLHF Training for Large Language Models with Stage Fusioncited by 43
  18. 2025Diversity-Aware Policy Optimization for Large Language Model Reasoningcited by 41
  19. 2025History Rhymes: Accelerating LLM Reinforcement Learning with RhymeRLcited by 38
  20. 2025Training Long-Context, Multi-Turn Software Engineering Agents with Reinforcement Learningcited by 36
  21. 2025RollPacker: Mitigating Long-Tail Rollouts for Fast, Synchronous RL Post-Trainingcited by 35
  22. 2025On-Policy RL with Optimal Reward Baselinecited by 34
  23. 2025Asymmetric REINFORCE for off-Policy Reinforcement Learning: Balancing positive and negative rewardscited by 29
  24. 2024Compute Or Load KV Cache? Why Not Both?cited by 25
  25. 2025Seer: Online Context Learning for Fast Synchronous LLM Reinforcement Learningcited by 25
  26. 2025Speculative Actions: A Lossless Framework for Faster Agentic Systemscited by 22
  27. 2025Simulating Rumor Spreading in Social Networks using LLM Agentscited by 16
  28. 2024KunServe: Parameter-centric Memory Management for Efficient Memory Overloading Handling in LLM Servingcited by 11
  29. 2025SpecEdge: Scalable Edge-Assisted Serving Framework for Interactive LLMscited by 10
  30. 2024PhoenixOS: Concurrent OS-level GPU Checkpoint and Restore with Validated Speculationcited by 6
  31. 2025MioHint: LLM-assisted Mutation for Whitebox API Testingcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2025RollArt: Disaggregated Multi-Task Agentic RL Training at Scalecited by 14
  2. 2026Learning to Explore with Parameter-Space Noise: A Deep Dive into Parameter-Space Noise for Reinforcement Learning with Verifiable Rewardscited by 6
  3. 2026TideRL: Boosting Agentic RL Goodput with Readiness-Aware Schedulingcited by 4
  4. 2026WeightBridge: An Efficient Weight Transfer Library for Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Accelerating Production LLMs with Combined Token/Embedding Speculators
  2. 2025FastMTP: Accelerating LLM Inference with Enhanced Multi-Token Prediction
  3. 2024Minions: Accelerating Large Language Model Inference with Aggregated Speculative Execution
  4. 2026LEAP: Learning Efficient Action Proposals For LLM Agents
  5. 2026Accelerating RL Post-Training Rollouts via System-Integrated Speculative Decoding
  6. 2024FastDraft: How to Train Your Draft
  7. 2025Beat the long tail: Distribution-Aware Speculative Decoding for RL Training
  8. 2025SpecEE: Accelerating Large Language Model Inference with Speculative Early Exiting