mlsanityData to 7 Oct 2026

CodeScaler: Scaling Code LLM Training and Test-Time Inference via Reward Models

arXiv 2602.17684 ↗First posted 4 Feb 2026cs.LG cs.AIcited by 5

On the map: LLM alignment / RLHF / preference optimisation → Process reward models for reasoning

Abstract (arXiv)

Reinforcement Learning from Verifiable Rewards (RLVR) has driven recent progress in code large language models by leveraging execution-based feedback from unit tests, but its scalability is fundamentally constrained by the availability and reliability of high-quality test cases. We propose CodeScaler, a reward model designed to scale both reinforcement learning training and test-time inference for code generation. CodeScaler is trained on carefully curated preference data derived from verified code problems and incorporates syntax-aware code extraction and validity-preserving reward shaping to ensure stable and robust optimization. Across four coding benchmarks, CodeScaler consistently outperforms execution-based RL by +1.55 points on Qwen3-8B-Base and +4.23 points on Qwen3-14B-Base. By further scaling to 44K problems with additional synthetic data, CodeScaler yields +14.64 points improvement over the base model without requiring any test cases. At inference time, CodeScaler serves as an effective test-time scaling method, achieving performance comparable to unit test approaches while providing a 10-fold reduction in latency. Moreover, CodeScaler surpasses existing reward models on RM-Bench not only in the code domain (+3.3 points), but also in general and reasoning domains (+2.7 points on average).

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Process reward models for reasoning, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 47 of its 56 references that are in the map. First posted on arXiv 4 Feb 2026.

Builds on

47 of 56 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025ACECODER: Acing Coder RL via Automated Test-Case Synthesis3 Feb
  2. 2024CodePMP: Scalable Preference Model Pretraining for Large Language Model Reasoning3 Oct

Other references · 45

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2021Program Synthesis with Large Language Modelscited by 3,509
  5. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  6. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
  7. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,628
  8. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
  9. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
  10. 2024Qwen2.5-Coder Technical Reportcited by 1,151
  11. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  12. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 890
33 more
  1. 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 638
  2. 2025LIMO: Less is More for Reasoningcited by 488
  3. 2022CodeT: Code Generation with Generated Testscited by 369
  4. 2023CodeT5+: Open Code Large Language Models for Code Understanding and Generationcited by 327
  5. 2023CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-Xcited by 294
  6. 2024LiveBench: A Challenging, Contamination-Limited LLM Benchmarkcited by 242
  7. 2025Skywork-Reward-V2: Scaling Preference Data Curation via Human-AI Synergycited by 204
  8. 2024RM-Bench: Benchmarking Reward Models of Language Models with Subtlety and Stylecited by 168
  9. 2023TACO: Topics in Algorithmic COde generation datasetcited by 150
  10. 2024RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learningcited by 146
  11. 2024MathScale: Scaling Instruction Tuning for Mathematical Reasoningcited by 144
  12. 2025KodCode: A Diverse, Challenging, and Verifiable Synthetic Dataset for Codingcited by 115
  13. 2025OpenCodeReasoning: Advancing Data Distillation for Competitive Codingcited by 106
  14. 2025Does Math Reasoning Improve General LLM Capabilities? Understanding Transferability of LLM Reasoningcited by 101
  15. 2023Execution-based Code Generation using Deep Reinforcement Learningcited by 95
  16. 2025S*: Test Time Scaling for Code Generationcited by 95
  17. 2023At Which Training Stage Does Code Data Help LLMs Reasoning?cited by 82
  18. 2024StepCoder: Improve Code Generation with Reinforcement Learning from Compiler Feedbackcited by 74
  19. 2025OpenCodeInstruct: A Large-scale Instruction Tuning Dataset for Code LLMscited by 57
  20. 2025Co-Evolving LLM Coder and Unit Tester via Reinforcement Learningcited by 51
  21. 2025rStar-Coder: Scaling Competitive Code Reasoning with a Large-Scale Verified Datasetcited by 49
  22. 2024EffiLearner: Enhancing Efficiency of Generated Code via Self-Optimizationcited by 28
  23. 2024How Does Code Pretraining Affect Language Model Task Performance?cited by 28
  24. 2026Large-Scale Terminal Agentic Trajectory Generation from Dockerized Environmentscited by 27
  25. 2023AceCoder: Utilizing Existing Code to Enhance Code Generationcited by 25
  26. 2024MHPP: Exploring the Capabilities and Limitations of Language Models Beyond Basic Code Generationcited by 25
  27. 2025EffiBench-X: A Multi-Language Benchmark for Measuring Efficiency of LLM-Generated Codecited by 24
  28. 2025CodeRL+: Improving Code Generation via Reinforcement with Execution Semantics Alignmentcited by 22
  29. 2025Can One Domain Help Others? A Data-Centric Study on Multi-Domain Reasoning via Reinforcement Learningcited by 17
  30. 2024EffiCoder: Enhancing Code Generation in Large Language Models through Efficiency-Aware Fine-tuningcited by 15
  31. 2024Sequence to Sequence Reward Modeling: Improving RLHF by Language Feedbackcited by 9
  32. 2025UnitCoder: Scalable Iterative Code Synthesis with Unit Test Guidancecited by 7
  33. 2025QueST: Incentivizing LLMs to Generate Difficult Problemscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026From Table to Cell: Attention for Better Reasoning with TABALIGNcited by 1
  2. 2026Themis: Training Robust Multilingual Code Reward Models for Flexible Multi-Criteria Scoringcited by 0
  3. 2026Mastermind: Strategy-grounded Learning for Repository-Scale Vulnerability Reproductioncited by 0
  4. 2026RLPF: Reinforcement Learning from Performance Feedback for Code Generationcited by 0
  5. 2026Test-Time Scaling in the Wild: Why Exploitation, Not Exploration, Is the Bottleneckcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Dynamic Scaling of Unit Tests for Code Reward Modeling
  2. 2025Rethinking Verification for LLM Code Generation: From Generation to Testing
  3. 2026Improving Small Language Models for Code Generation with Reinforcement Learning from Verification Feedback
  4. 2026Scaling Agentic Verifier for Competitive Coding
  5. 2026Information-Gain Rewards over Diversity-Pruned Tests: GT-Anchored Verifier Co-Training for Reliable Code Generation
  6. 2026Enhancing the Code Reasoning Capabilities of LLMs via Consistency-based Reinforcement Learning
  7. 2025Afterburner: Reinforcement Learning Facilitates Self-Improving Code Efficiency Optimization
  8. 2025Scaling Up RL: Unlocking Diverse Reasoning in LLMs via Prolonged Training