mlsanityData to 7 Oct 2026

LLMLOOP: Improving LLM-Generated Code and Tests through Automated Iterative Feedback Loops

arXiv 2603.23613 ↗First posted 24 Mar 2026cs.SE cs.AIcited by 8

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) are showing remarkable performance in generating source code, yet the generated code often has issues like compilation errors or incorrect code. Researchers and developers often face wasted effort in implementing checks and refining LLM-generated code, frequently duplicating their efforts. This paper presents LLMLOOP, a framework that automates the refinement of both source code and test cases produced by LLMs. LLMLOOP employs five iterative loops: resolving compilation errors, addressing static analysis issues, fixing test case failures, and improving test quality through mutation analysis. These loops ensure the generation of high-quality test cases that serve as both a validation mechanism and a regression test suite for the generated code. We evaluated LLMLOOP on HUMANEVAL-X, a recent benchmark of programming tasks. Results demonstrate the tool's effectiveness in refining LLM-generated outputs.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 12 of its 22 references that are in the map. First posted on arXiv 24 Mar 2026.

Builds on

12 of 22 references.

References · 12

  1. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  2. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,332
  3. 2023CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-Xcited by 294
  4. 2023An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generationcited by 193
  5. 2022Large Language Models are Few-shot Testers: Exploring LLM-based General Bug Reproductioncited by 110
  6. 2022Pynguin: Automated Unit Test Generation for Pythoncited by 49
  7. 2024The Future of AI-Driven Software Engineeringcited by 23
  8. 2023LLM for Test Script Generation and Migration: Challenges, Capabilities, and Opportunitiescited by 19
  9. 2024Do LLMs generate test oracles that capture the actual or the expected program behaviour?cited by 10
  10. 2024Comparing large language models and human programmers for generating programming codecited by 9
  11. 2021Automatic Code Generation using Pre-Trained Language Modelscited by 8
  12. 2024An Empirical Study on Self-correcting Large Language Models for Data Science Code Generationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 8.

8 papers.

  1. 2026Beyond Fail-to-Pass: Iterative Hardening of Co-Generated Bug Reproduction Tests and Fixescited by 2
  2. 2026VeriSimpl: Robust Optimization Modeling from Natural Language using Simplification-based Verificationcited by 2
  3. 2026MR-Coupler: Automated Metamorphic Test Generation via Functional Coupling Analysiscited by 1
  4. 2026Natural-Language to SysMLv2 Translation via Conformance-Driven Iterative Refinementcited by 0
  5. 2026Comprendia: AI-Augmented Code Comprehensioncited by 0
  6. 2026ExecCritic: Learn to Test, Test to Improve for Coding Agentscited by 0
  7. 2026Segment-Level Agentic Topic Modeling for Improved Data Exploration and Resource Efficiencycited by 0
  8. 2026ReMCTS: Reflection-Enhanced Monte Carlo Tree Search for Code Generationcited by 0