# RefineRL: Advancing Competitive Programming with Self-Refinement Reinforcement Learning

arXiv: https://arxiv.org/abs/2604.00790 · HTML: https://mlsanity.com/paper/2604.00790/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning.
- Builds on 30 of its 35 references that are in the map.
- First posted on arXiv 1 Apr 2026.

## Placed in

- LLM reasoning / [Reinforcement learning for reasoning](https://mlsanity.com/lr/reinforcement-learning-for-reasoning/)

## Builds on (most cited first, 30 of 30)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [HybridFlow: A Flexible and Efficient RLHF Framework](https://mlsanity.com/paper/2409.19256/) (2024)
- [LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code](https://mlsanity.com/paper/2403.07974/) (2024)
- [Teaching Large Language Models to Self-Debug](https://mlsanity.com/paper/2304.05128/) (2023)
- [AlphaEvolve: A coding agent for scientific and algorithmic discovery](https://mlsanity.com/paper/2506.13131/) (2025)
- [Training Language Models to Self-Correct via Reinforcement Learning](https://mlsanity.com/paper/2409.12917/) (2024)
- [Goedel-Prover-V2: Scaling Formal Theorem Proving with Scaffolded Data Synthesis and Self-Correction](https://mlsanity.com/paper/2508.03613/) (2025)
- [Competitive Programming with Large Reasoning Models](https://mlsanity.com/paper/2502.06807/) (2025)
- [OpenCodeReasoning: Advancing Data Distillation for Competitive Coding](https://mlsanity.com/paper/2504.01943/) (2025)
- [Seed-Prover: Deep and Broad Reasoning for Automated Theorem Proving](https://mlsanity.com/paper/2507.23726/) (2025)
- [S*: Test Time Scaling for Code Generation](https://mlsanity.com/paper/2502.14382/) (2025)
- [Gold-medalist Performance in Solving Olympiad Geometry with AlphaGeometry2](https://mlsanity.com/paper/2502.03544/) (2025)
- [Can Language Models Solve Olympiad Programming?](https://mlsanity.com/paper/2404.10952/) (2024)
- [SETS: Leveraging Self-Verification and Self-Correction for Improved Test-Time Scaling](https://mlsanity.com/paper/2501.19306/) (2025)
- [CodeContests+: High-Quality Test Case Generation for Competitive Programming](https://mlsanity.com/paper/2506.05817/) (2025)
- [Z1: Efficient Test-time Scaling with Code](https://mlsanity.com/paper/2504.00810/) (2025)
- [Solving Formal Math Problems by Decomposition and Iterative Reflection](https://mlsanity.com/paper/2507.15225/) (2025)
- [Multi-Turn Code Generation Through Single-Step Rewards](https://mlsanity.com/paper/2502.20380/) (2025)
- [HardTests: Synthesizing High-Quality Test Cases for LLM Coding](https://mlsanity.com/paper/2505.24098/) (2025)
- [ProgCo: Program Helps Self-Correction of Large Language Models](https://mlsanity.com/paper/2501.01264/) (2025)
- [StepFun-Prover Preview: Let's Think and Verify Step by Step](https://mlsanity.com/paper/2507.20199/) (2025)
- [Scaling Test-Time Compute to Achieve IOI Gold Medal with Open-Weight Models](https://mlsanity.com/paper/2510.14232/) (2025)
- [Klear-CodeTest: Scalable Test Case Generation for Code Reinforcement Learning](https://mlsanity.com/paper/2508.05710/) (2025)
- [Scaling Agentic Verifier for Competitive Coding](https://mlsanity.com/paper/2602.04254/) (2026)
- [ATGen: Adversarial Reinforcement Learning for Test Case Generation](https://mlsanity.com/paper/2510.14635/) (2025)
- [QueST: Incentivizing LLMs to Generate Difficult Problems](https://mlsanity.com/paper/2510.17715/) (2025)

## Built on by (most cited first, 2 of 2)

- [FC-SWE: Failure-Conditioned RL for Long-Horizon Software Engineering Agents](https://mlsanity.com/paper/2610.07898/) (2026)
- [Beyond the Verdict: Agentic Code Judging with Verifiable Feedback](https://mlsanity.com/paper/2610.18760/) (2026)

## Abstract (arXiv)

While large language models (LLMs) have demonstrated strong performance on complex reasoning tasks such as competitive programming (CP), existing methods predominantly focus on single-attempt settings, overlooking their capacity for iterative refinement. In this paper, we present RefineRL, a novel approach designed to unleash the self-refinement capabilities of LLMs for CP problem solving. RefineRL introduces two key innovations: (1) Skeptical-Agent, an iterative self-refinement agent equipped with local execution tools to validate generated solutions against public test cases of CP problems. This agent always remains skeptical of its own outputs and thereby enforces rigorous self-refinement even when validation suggests correctness. (2) A reinforcement learning (RL) solution to incentivize LLMs to self-refine with only standard RLVR data (i.e., problems paired with their verifiable answers). Extensive experiments on Qwen3-4B and Qwen3-4B-2507 demonstrate that our method yields substantial gains: after our RL training, these compact 4B models integrated with the Skeptical-Agent not only outperform much larger 32B models but also approach the single-attempt performance of 235B models. Notably, although our models are trained solely on CP coding data, RefineRL also substantially improves their performance on mathematical reasoning benchmarks, suggesting that the learned self-refinement behavior can transfer beyond coding-specific tasks. These findings highlight self-refinement as a promising direction for scaling LLMs' general reasoning ability.