# Extrapolative Weight Averaging Reveals Correctness-Efficiency Frontiers in Code RL

arXiv: https://arxiv.org/abs/2605.28751 · HTML: https://mlsanity.com/paper/2605.28751/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 49 references that are in the map.
- First posted on arXiv 27 May 2026.

## Builds on (most cited first, 25 of 25)

- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [DAPO: An Open-Source LLM Reinforcement Learning System at Scale](https://mlsanity.com/paper/2503.14476/) (2025)
- [Understanding R1-Zero-Like Training: A Critical Perspective](https://mlsanity.com/paper/2503.20783/) (2025)
- [Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?](https://mlsanity.com/paper/2504.13837/) (2025)
- [Kimi K2: Open Agentic Intelligence](https://mlsanity.com/paper/2507.20534/) (2025)
- [The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models](https://mlsanity.com/paper/2505.22617/) (2025)
- [MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attention](https://mlsanity.com/paper/2506.13585/) (2025)
- [AIMO-2 Winning Solution: Building State-of-the-Art Mathematical Reasoning Models with OpenMathReasoning dataset](https://mlsanity.com/paper/2504.16891/) (2025)
- [TACO: Topics in Algorithmic COde generation dataset](https://mlsanity.com/paper/2312.14852/) (2023)
- [SimpleTIR: End-to-End Reinforcement Learning for Multi-Turn Tool-Integrated Reasoning](https://mlsanity.com/paper/2509.02479/) (2025)
- [The Invisible Leash: Why RLVR May or May Not Escape Its Origin](https://mlsanity.com/paper/2507.14843/) (2025)
- [CWM: An Open-Weights LLM for Research on Code Generation with World Models](https://mlsanity.com/paper/2510.02387/) (2025)
- [CodeContests+: High-Quality Test Case Generation for Competitive Programming](https://mlsanity.com/paper/2506.05817/) (2025)
- [WARP: On the Benefits of Weight Averaged Rewarded Policies](https://mlsanity.com/paper/2406.16768/) (2024)
- [Neural Thickets: Diverse Task Experts Are Dense Around Pretrained Weights](https://mlsanity.com/paper/2603.12228/) (2026)
- [Process Supervision-Guided Policy Optimization for Code Generation](https://mlsanity.com/paper/2410.17621/) (2024)
- [OpenCodeReasoning-II: A Simple Test Time Scaling Approach via Self-Critique](https://mlsanity.com/paper/2507.09075/) (2025)
- [On Predictability of Reinforcement Learning Dynamics for Large Language Models](https://mlsanity.com/paper/2510.00553/) (2025)
- [SWE-fficiency: Can Language Models Optimize Real-World Repositories on Real Workloads?](https://mlsanity.com/paper/2511.06090/) (2025)
- [Afterburner: Reinforcement Learning Facilitates Self-Improving Code Efficiency Optimization](https://mlsanity.com/paper/2505.23387/) (2025)
- [ReVeal: Self-Evolving Code Agents via Reliable Self-Verification](https://mlsanity.com/paper/2506.11442/) (2025)
- [Scaling Test-Time Compute for Agentic Coding](https://mlsanity.com/paper/2604.16529/) (2026)
- [BigO(Bench): Can LLMs Generate Code with Controlled Time and Space Complexity?](https://mlsanity.com/paper/2503.15242/) (2025)
- [Linear Dynamics in the RLVR Training of Large Language Models](https://mlsanity.com/paper/2601.04537/) (2026)
- [Self-Execution Simulation Improves Coding Models](https://mlsanity.com/paper/2604.03253/) (2026)

## Built on by (most cited first, 2 of 2)

- [Reinforcement Learning for Code Optimization](https://mlsanity.com/paper/2607.25970/) (2026)
- [Does Scaling Reinforcement Learning Really Require More Training?](https://mlsanity.com/paper/2610.01133/) (2026)

## Abstract (arXiv)

Linear interpolation between fine-tuned checkpoints has been shown to trace the Pareto front between competing objectives, but whether extrapolative weight averaging can extend such frontiers to new checkpoints useful at inference time, without additional RL training, remains unclear. We study this question in RL for competitive programming, where hidden unit tests under time and memory limits enforce both functional correctness and computational efficiency. Starting from a shared initialization, we train checkpoints under nested unit-test coverage: low-coverage rewards require passing smaller-input tests, while high-coverage rewards require passing progressively larger tests up to the full suite. This sweep reveals the emergence of a correctness-efficiency frontier: on hard problems, higher-coverage reward reduces optimization failures but increases correctness failures, leaving solve rate nearly unchanged. Interpolation between low- and high-coverage checkpoints recovers this frontier, while extrapolation extends it beyond the trained endpoints. Both the frontier and its extrapolative continuation appear across three inference settings, pure reasoning, tool use, and agentic coding, and across two model scales, 32B and 7B. At the problem level, moving along the frontier changes which problems are solved, making extrapolated checkpoints complementary policies in inference-time scaling. Ensembles with extrapolative weight averaging broaden coverage and improve pass@250 on LCB/hard by 3.3% over the best single checkpoint at matched sample budget. These results show that nested unit-test coverage in code RL induces a frontier that extrapolative weight averaging can navigate, extend, and exploit.