mlsanityData to 7 Oct 2026

Dynamic Scaling of Unit Tests for Code Reward Modeling

arXiv 2501.01054 ↗First posted 2 Jan 2025cs.CL cs.SEcited by 31

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

Current large language models (LLMs) often struggle to produce accurate responses on the first attempt for complex reasoning tasks like code generation. Prior research tackles this challenge by generating multiple candidate solutions and validating them with LLM-generated unit tests. The execution results of unit tests serve as reward signals to identify correct solutions. As LLMs always confidently make mistakes, these unit tests are not reliable, thereby diminishing the quality of reward signals. Motivated by the observation that scaling the number of solutions improves LLM performance, we explore the impact of scaling unit tests to enhance reward signal quality. Our pioneer experiment reveals a positive correlation between the number of unit tests and reward signal quality, with greater benefits observed in more challenging problems. Based on these insights, we propose CodeRM-8B, a lightweight yet effective unit test generator that enables efficient and high-quality unit test scaling. Additionally, we implement a dynamic scaling mechanism that adapts the number of unit tests based on problem difficulty, further improving efficiency. Experimental results show that our approach significantly improves performance across various models on three benchmarks (e.g., with gains of 18.43% for Llama3-8B and 3.42% for GPT-4o-mini on HumanEval Plus).

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 13 of its 44 references that are in the map. First posted on arXiv 2 Jan 2025.

Builds on

13 of 44 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code12 Mar

Other references · 12

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  6. 2021Program Synthesis with Large Language Modelscited by 3,509
  7. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  8. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  9. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 946
  10. 2023TACO: Topics in Algorithmic COde generation datasetcited by 150
  11. 2024Learning How Hard to Think: Input-Adaptive Allocation of LM Computationcited by 67
  12. 2024Smaller, Weaker, Yet Better: Training LLM Reasoners via Compute-Optimal Samplingcited by 60

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 11; 2026: 20.

31 papers.

Sort:

By area:

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  2. 2025d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learningcited by 178
  3. 2025Co-Evolving LLM Coder and Unit Tester via Reinforcement Learningcited by 51
  4. 2025MCTS-Judge: Test-Time Scaling in LLM-as-a-Judge for Code Correctness Evaluationcited by 32
  5. 2026AutoSOTA: An End-to-End Automated Research System for State-of-the-Art AI Model Discoverycited by 17
  6. 2025A Practitioner's Guide to Multi-turn Agentic Reinforcement Learningcited by 16
  7. 2025Incentivizing Agentic Reasoning in LLM Judges via Tool-Integrated Reinforcement Learningcited by 16
  8. 2026SWE-Universe: Scale Real-World Verifiable Environments to Millionscited by 16
  9. 2025Rethinking Verification for LLM Code Generation: From Generation to Testingcited by 12
  10. 2025Learning to Generate Unit Test via Adversarial Reinforcement Learningcited by 11
  11. 2025Teaching Your Models to Understand Code via Focal Preference Alignmentcited by 10
  12. 2025Scoring Verifiers: Evaluating Synthetic Verification for Code and Reasoningcited by 8
19 more
  1. 2026Scaling Coding Agents via Atomic Skillscited by 8
  2. 2026Scaling Agentic Verifier for Competitive Codingcited by 5
  3. 2025MultiFileTest: A Multi-File-Level LLM Unit Test Generation Benchmark and Impact of Error Fixing Mechanismscited by 4
  4. 2026Rule2DRC: Benchmarking LLM Agents for DRC Script Synthesis with Execution-Guided Test Generationcited by 2
  5. 2026Self-Improving Code Generation via Semantic Entropy and Behavioral Consensuscited by 1
  6. 2026Adaptive Generate-Rank-Verify: Inference-Time Search with Costly Verificationcited by 1
  7. 2026Granularity-Regulated Adaptive Computational Efficiency for Optimal Verification in Test-Time Scalingcited by 1
  8. 2026VeriBound: PAC-Bayesian Generalization Bounds for Process Reward Models Trained with Formal Verification Toolscited by 1
  9. 2026The geometry of AI validation: From structural blindness to reusable auditscited by 1
  10. 2026Two-Stage Reinforcement Learning for Sound and Adversarial Test Generation in Code LLMscited by 1
  11. 2026CVeDRL: An Efficient Code Verifier via Difficulty-aware Reinforcement Learningcited by 0
  12. 2026Synthesizing File-Level Data for Unit Test Generation with Chain-of-Thoughts via Self-Debuggingcited by 0
  13. 2026MIST-RL: Mutation-based Incremental Suite Testing via Reinforcement Learningcited by 0
  14. 2026Improving Code Translation with Syntax-Guided and Semantic-aware Preference Optimizationcited by 0
  15. 2026The Generalization Spectrum: A Chromatographic Approach to Evaluating Learning Algorithmscited by 0
  16. 2026Function-Level Execution Feedback for Code Preference Optimizationcited by 0
  17. 2026Robust Code RL via Faulty-Code-Driven Test case Synthesis and Dense Reward Shapingcited by 0
  18. 2026Confidence-Gated Transductive Test Generation for Code Rerankingcited by 0
  19. 2026Cheap to Draw, Expensive to Trust: Certifying Test-Time Scaling Curvescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026CodeScaler: Scaling Code LLM Training and Test-Time Inference via Reward Models
  2. 2025From Mathematical Reasoning to Code: Generalization of Process Reward Models in Test-Time Scaling
  3. 2025Rethinking Reward Models for Multi-Domain Test-Time Scaling
  4. 2025Smaller Models, Smarter Rewards: A Two-Sided Approach to Process and Outcome Rewards
  5. 2025S*: Test Time Scaling for Code Generation
  6. 2026Sample Count Is Not Enough: Candidate-Generation Strategy Shapes the Energy and Performance of LLM Test-Time Scaling
  7. 2025Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scaling
  8. 2026FunPRM: Function-as-Step Process Reward Model with Meta Reward Correction for Code Generation