Miles v0.1: Production-Level Post-Training
Not on the map. Why?
Abstract (arXiv)
We present Miles v0.1, a full-stack, production-ready system for frontier post-training. Building upon the clean design of slime, Miles designs each stage of the reinforcement-learning (RL) training loop around a single principle: components should be verified, clean, and customizable. With accuracy, efficiency, reliability, and scalability as first-class goals, Miles aims to make frontier-scale RL accessible to researchers and enterprises alike. This report walks through the system end to end: rollout engines built on SGLang, a trainer with a choice of two backends (NVIDIA Megatron-LM and PyTorch FSDP), and three weight-synchronization transports for different deployment topologies. Beyond full-parameter RL, Miles also supports LoRA RL, on-policy distillation, supervised fine-tuning, and true-on-policy rollout-training alignment, and extends the same architecture to diffusion models. We close with an end-to-end case study: fully asynchronous agentic RL on a GLM-5.2 744B-A40B model over terminal-use coding tasks, running on 64 NVIDIA GB300 GPUs with a median step time of 263 seconds over the first 30 measured steps. Miles is open-sourced at https://github.com/radixark/miles, with the project website at https://miles.radixark.com.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 27 of its 59 references that are in the map. First posted on arXiv 8 Sep 2026.
Builds on
27 of 59 references.
References · 27
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2025Qwen3 Technical Reportcited by 8,820
- 2024Qwen2.5 Technical Reportcited by 4,942
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
- 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
- 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,402
- 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,386
- 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
- 2025Qwen-Image Technical Reportcited by 1,044
15 more
- 2025Group Sequence Policy Optimizationcited by 835
- 2025DeepSeek-V3.2: Pushing the Frontier of Open Large Language Modelscited by 815
- 2025Flow-GRPO: Training Flow Matching Models via Online RLcited by 661
- 2023PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallelcited by 660
- 2025Kimi K2: Open Agentic Intelligencecited by 567
- 2024FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precisioncited by 519
- 2026Kimi K2.5: Visual Agentic Intelligencecited by 501
- 2015Policy Distillationcited by 440
- 2026Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipecited by 299
- 2025REINFORCE++: Stabilizing Critic-Free Policy Optimization with Global Advantage Normalizationcited by 241
- 2025DiffusionNFT: Online Diffusion Reinforcement with Forward Processcited by 230
- 2026Cosmos 3: Omnimodal World Models for Physical AIcited by 165
- 2025Stabilizing MoE Reinforcement Learning by Aligning Training and Inference Routerscited by 72
- 2024USP: A Unified Sequence Parallelism Approach for Long Context Generative AIcited by 61
- 2025Four Over Six: More Accurate NVFP4 Quantization with Adaptive Block Scalingcited by 35
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025Laminar: A Scalable Asynchronous RL Post-Training Framework
- 2026Where Should RL Post-Training Compute Go? Model Size, Search, Learning, and Feedback
- 2025Part II: ROLL Flash -- Accelerating RLVR and Agentic Training with Asynchrony
- 2026RL Excursions during Pre-Training: Re-examining Policy Optimization for LLM training
- 2026QwenGyre: An Elastic Reinforcement Learning Framework for Training xLong-Horizon Agents
- 2025RollPacker: Mitigating Long-Tail Rollouts for Fast, Synchronous RL Post-Training
- 2026Libra: Efficient Resource Management for Agentic RL Post-Training
- 2026Neglected Free Lunch from Post-training: Progress Advantage for LLM Agents