mlsanityData to 14 Oct 2026

CodeLift: Your Test Suite Is Secretly a Dense SWE Curriculum

arXiv 2610.17224 ↗First posted 12 Oct 2026cs.LG cs.SEcited by 0

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

The pass-all-test binary reward has become a standard recipe for training software engineering agents, but its sparsity eliminates difference of partially correct patches. The test pass-rate is a natural alternative. However, we find that it is not a reliable indicator of patch quality. A functionality being tested can be covered by either one challenging test or one hundred trivial tests, where the majority might be already satisfied pass-to-pass tests. We introduce CodeLift, a reinforcement learning method that dynamically calibrates pass-rate rewards, turning the test suite into an adaptive curriculum. At each iteration, CodeLift groups tests that kill the same patches and patches that pass the same tests, then gives more weight to test groups that fewer patch groups pass. This reduces the impact of majority simple tests and focuses more on what is still challenging. Within each rollout, CodeLift applies the same scoring rule to repository snapshots, assigning credit to useful actions even when later ones introduce regressions. Experiments on SWE benchmarks show that CodeLift consistently improves pass@1/2/4/8 over binary and pass-rate baselines. Analyses show that CodeLift provides dense, stable learning signals and unlocks the inherent curriculum in the test suite.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 14 of its 18 references that are in the map. First posted on arXiv 12 Oct 2026.

Builds on

14 of 18 references.

Closest ancestors · 4

Same direction, newest first

  1. 2026GLM-5: from Vibe Coding to Agentic Engineering17 Feb
  2. 2025SWE-rebench: An Automated Pipeline for Task Collection and Decontaminated Evaluation of Software Engineering Agents26 May
  3. 2025R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agents9 Apr
  4. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering6 May

Other references · 10

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  4. 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 1,040
  5. 2025Group-in-Group Policy Optimization for LLM Agent Trainingcited by 497
  6. 2025MiMo: Unlocking the Reasoning Potential of Language Model -- From Pretraining to Posttrainingcited by 121
  7. 2025TreeRL: LLM Reinforcement Learning with On-Policy Tree Searchcited by 80
  8. 2026Branching Policy Optimization: Sandbox-Native Language Agent Reinforcement Learningcited by 12
  9. 2026Beyond Binary: Turning Partial Success into Dense Verifiable Rewards for Reinforcement Learning in Code Generationcited by 5
  10. 2026Exploring Pass-Rate Reward in Reinforcement Learning for Code Generationcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026SWEPatchTest: Evaluating Software Tests Beyond Reproduction
  2. 2025SWE-RM: Execution-free Feedback For Software Engineering Agents
  3. 2026SWE-Replay: Efficient Test-Time Scaling for Software Engineering Agents
  4. 2026SWE-Mutation: Can LLMs Generate Reliable Test Suites in Software Engineering?
  5. 2026ExecCritic: Learn to Test, Test to Improve for Coding Agents
  6. 2026TestJack: Should you trust the results in coding benchmarks? Agentic Coding Benchmarks Auditing via Evaluator Evolution
  7. 2026Breaking, Stale, or Missing? Benchmarking Coding Agents on Project-Level Test Evolution
  8. 2026TaPR: Test-Aware Policy Refinement for Feedback-Conditioned Code Generation