mlsanityData to 7 Oct 2026

MIST-RL: Mutation-based Incremental Suite Testing via Reinforcement Learning

arXiv 2603.01409 ↗First posted 2 Mar 2026cs.AI cs.LG cs.SEcited by 0

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) often fail to generate correct code on the first attempt, which requires using generated unit tests as verifiers to validate the solutions. Despite the success of recent verification methods, they remain constrained by a "scaling-by-quantity" paradigm. This brute-force approach suffers from a critical limitation: it yields diminishing returns in fault detection while causing severe test redundancy. To address this, we propose MIST-RL (Mutation-based Incremental Suite Testing via Reinforcement Learning), a framework that shifts the focus to "scaling-by-utility". We formulate test generation as a sequential decision process optimized via Group Relative Policy Optimization (GRPO). Specifically, we introduce a novel incremental mutation reward combined with dynamic penalties, which incentivizes the model to discover new faults while it suppresses functionally equivalent assertions. Experiments on HumanEval+ and MBPP+ demonstrate that MIST-RL outperforms state-of-the-art baselines. It achieves a +28.5% higher mutation score while reducing the number of test cases by 19.3%. Furthermore, we show that these compact, high-utility tests serve as superior verifiers, which improves downstream code reranking accuracy on HumanEval+ by 3.05% over the SOTA baseline with 10 candidate samples. The source code and data are provided in the supplementary material.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 25 of its 39 references that are in the map. First posted on arXiv 2 Mar 2026.

Builds on

25 of 39 references.

References · 25

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  6. 2025Qwen3 Technical Reportcited by 8,195
  7. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  8. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  9. 2021Program Synthesis with Large Language Modelscited by 3,509
  10. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  11. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
  12. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,332
13 more
  1. 2022Scaling Laws for Reward Model Overoptimizationcited by 1,056
  2. 2022DS-1000: A Natural and Reliable Benchmark for Data Science Code Generationcited by 442
  3. 2022CodeT: Code Generation with Generated Testscited by 369
  4. 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 342
  5. 2023An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generationcited by 193
  6. 2022Natural Language to Code Translation with Executioncited by 122
  7. 2023RLTF: Reinforcement Learning from Unit Test Feedbackcited by 110
  8. 2023Execution-based Code Generation using Deep Reinforcement Learningcited by 95
  9. 2022Interactive Code Generation via Test-Driven User-Intent Formalizationcited by 50
  10. 2022Pynguin: Automated Unit Test Generation for Pythoncited by 49
  11. 2023Enhancing Large Language Models in Coding Through Multi-Perspective Self-Consistencycited by 46
  12. 2022Fault-Aware Neural Code Rankerscited by 33
  13. 2025Dynamic Scaling of Unit Tests for Code Reward Modelingcited by 31

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026SWE-Mutation: Can LLMs Generate Reliable Test Suites in Software Engineering?
  2. 2024Reinforcement Learning from Automatic Feedback for High-Quality Unit Test Generation
  3. 2023Reinforcement Learning from Automatic Feedback for High-Quality Unit Test Generation
  4. 2026GRPO with State Mutations: Improving LLM-Based Hardware Test Plan Generation
  5. 2026TaPR: Test-Aware Policy Refinement for Feedback-Conditioned Code Generation
  6. 2025Co-Evolving LLM Coder and Unit Tester via Reinforcement Learning
  7. 2025TestForge: Feedback-Driven, Agentic Test Suite Generation
  8. 2025HarnessLLM: Automatic Testing Harness Generation via Reinforcement Learning