# Noisy Test-Time Reinforcement Learning for Code LLMs

arXiv: https://arxiv.org/abs/2609.32172 · HTML: https://mlsanity.com/paper/2609.32172/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 42 references that are in the map.
- First posted on arXiv 26 Sep 2026.

## Builds on (most cited first, 24 of 24)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [Qwen2.5-Coder Technical Report](https://mlsanity.com/paper/2409.12186/) (2024)
- [DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence](https://mlsanity.com/paper/2401.14196/) (2024)
- [BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions](https://mlsanity.com/paper/2406.15877/) (2024)
- [PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Prompts](https://mlsanity.com/paper/2306.04528/) (2023)
- [Right Question is Already Half the Answer: Fully Unsupervised LLM Reasoning Incentivization](https://mlsanity.com/paper/2504.05812/) (2025)
- [Improving the Robustness of Large Language Models via Consistency Alignment](https://mlsanity.com/paper/2403.14221/) (2024)
- [A note on the triangle inequality for the Jaccard distance](https://mlsanity.com/paper/1612.02696/) (2016)
- [Robustness, Security, Privacy, Explainability, Efficiency, and Usability of Large Language Models for Code](https://mlsanity.com/paper/2403.07506/) (2024)
- [HumanEvalComm: Benchmarking the Communication Competence of Code Generation for LLMs and LLM Agent](https://mlsanity.com/paper/2406.00215/) (2024)
- [Certified Robustness for Large Language Models with Self-Denoising](https://mlsanity.com/paper/2307.07171/) (2023)
- [Dynamic Benchmarking of Reasoning Capabilities in Code Large Language Models Under Data Contamination](https://mlsanity.com/paper/2503.04149/) (2025)
- [NLPerturbator: Studying the Robustness of Code LLMs to Natural Language Variations](https://mlsanity.com/paper/2406.19783/) (2024)
- [Adversarial Training of Reward Models](https://mlsanity.com/paper/2504.06141/) (2025)
- [Towards Understanding the Robustness of LLM-based Evaluations under Perturbations](https://mlsanity.com/paper/2412.09269/) (2024)
- [On the Limitations of Embedding Based Methods for Measuring Functional Correctness for Code Generation](https://mlsanity.com/paper/2405.01580/) (2024)
- [SAGE: A Realistic Benchmark for Semantic Understanding](https://mlsanity.com/paper/2509.21310/) (2025)
- [RobuNFR: Evaluating the Robustness of Large Language Models on Non-Functional Requirements Aware Code Generation](https://mlsanity.com/paper/2503.22851/) (2025)
- [CREME: Robustness Enhancement of Code LLMs via Layer-Aware Model Editing](https://mlsanity.com/paper/2507.16407/) (2025)
- [Code Roulette: How Prompt Variability Affects LLM Code Generation](https://mlsanity.com/paper/2506.10204/) (2025)
- [On the Adversarial Robustness of Instruction-Tuned Large Language Models for Code](https://mlsanity.com/paper/2411.19508/) (2024)
- [Adversarial Attack Classification and Robustness Testing for Large Language Models for Code](https://mlsanity.com/paper/2506.07942/) (2025)

## Abstract (arXiv)

Large language models (LLMs) have demonstrated remarkable performance across various code-related tasks. However, unlike carefully curated datasets that are typically high-quality and error-free, real-world user instructions are often vague and error-prone, posing significant challenges to the robustness of code LLMs. Furthermore, robustness-oriented fine-tuning relies on paired clean-noisy samples, which are costly to curate and require sophisticated noisy simulation techniques. To address these challenges, we propose the Noisy Test-time Reinforcement Learning framework (NTRL-Code), which enables robust self-evolution of code LLMs using only unlabeled noisy data during the testing stage. Specifically, NTRL-Code uses conservative self-denoising to obtain a cleaner semantic anchor for target estimation, and employs an abstract-syntax-tree (AST)-based structural aggregation mechanism to estimate a proxy target from multiple candidate programs. The policy is then optimized on the original noisy prompts with a hybrid reward that combines format validity, code similarity, and anti-repetition signals. Extensive experiments on three benchmarks, each incorporating character-level, word-level, and paragraph-level perturbations, demonstrate that NTRL-Code yields robust and consistent improvements, stabilizing the predictions of various base models. Our code is available at https://github.com/Xikai97/NTRL-Code.