mlsanityData to 7 Oct 2026

debug-gym: A Text-Based Environment for Interactive Debugging

arXiv 2503.21557 ↗First posted 27 Mar 2025cs.AI cs.CL cs.PLcited by 19

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

Large Language Models (LLMs) are increasingly relied upon for coding tasks, yet in most scenarios it is assumed that all relevant information can be either accessed in context or matches their training data. We posit that LLMs can benefit from the ability to interactively explore a codebase to gather the information relevant to their task. To achieve this, we present a textual environment, namely debug-gym, for developing LLM-based agents in an interactive coding setting. Our environment is lightweight and provides a preset of useful tools, such as a Python debugger (pdb), designed to facilitate an LLM-based agent's interactive debugging. Beyond coding and debugging tasks, this approach can be generalized to other tasks that would benefit from information-seeking behavior by an LLM agent.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 64 of its 85 references that are in the map. First posted on arXiv 27 Mar 2025.

Builds on

64 of 85 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024Training Software Engineering Agents and Verifiers with SWE-Gym30 Dec
  2. 2024Agentless: Demystifying LLM-based Software Engineering Agents1 Jul
  3. 2024MASAI: Modular Architecture for Software-engineering AI Agents17 Jun
  4. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering6 May
  5. 2024Executable Code Actions Elicit Better LLM Agents1 Feb
  6. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation16 Aug

Other references · 58

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  7. 2021Program Synthesis with Large Language Modelscited by 3,509
  8. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  9. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  10. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  11. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
  12. 2024A Survey on LLM-as-a-Judgecited by 1,384
38 more
  1. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,134
  2. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
  3. 2023Teaching Large Language Models to Self-Debugcited by 963
  4. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 895
  5. 2021Measuring Coding Challenge Competence With APPScited by 879
  6. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 876
  7. 2024A Survey on Large Language Models for Code Generationcited by 646
  8. 2021Unified Pre-training for Program Understanding and Generationcited by 361
  9. 2023RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generationcited by 357
  10. 2018TextWorld: A Learning Environment for Text-based Gamescited by 304
  11. 2022ScienceWorld: Is your Agent Smarter than a 5th Grader?cited by 298
  12. 2024AutoCodeRover: Autonomous Program Improvementcited by 272
  13. 2024OpenCodeInterpreter: Integrating Code Generation with Execution and Refinementcited by 242
  14. 2024V-STaR: Training Verifiers for Self-Taught Reasonerscited by 214
  15. 2023Is Self-Repair a Silver Bullet for Code Generation?cited by 191
  16. 2019Interactive Fiction Games: A Colossal Adventurecited by 165
  17. 2023CodePlan: Repository-level Coding using LLMs and Planningcited by 139
  18. 2019Learning to Speak and Act in a Fantasy Text Adventure Gamecited by 122
  19. 2023Self-Edit: Fault-Aware Code Editor for Code Generationcited by 110
  20. 2024Code Generation with AlphaCodium: From Prompt Engineering to Flow Engineeringcited by 100
  21. 2024WorldCoder, a Model-Based LLM Agent: Building World Models by Writing Code and Interacting with the Environmentcited by 99
  22. 2024SWE-Bench+: Enhanced Coding Benchmark for LLMscited by 95
  23. 2024DISCOVERYWORLD: A Virtual Environment for Developing and Evaluating Automated Scientific Discovery Agentscited by 89
  24. 2024DebugBench: Evaluating Debugging Capability of Large Language Modelscited by 74
  25. 2024Automated Unit Test Improvement using Large Language Models at Metacited by 73
  26. 2015Language Understanding for Text-based Games Using Deep Reinforcement Learningcited by 70
  27. 2024LLM-Based Test-Driven Interactive Code Generation: User Study and Empirical Evaluationcited by 46
  28. 2024From Code to Correctness: Closing the Last Mile of Code Generation with Hierarchical Debuggingcited by 46
  29. 2023SelfEvolve: A Code Evolution Framework via Large Language Modelscited by 43
  30. 2024CYCLE: Learning to Self-Refine the Code Generationcited by 39
  31. 2024A Systematic Literature Review on Large Language Models for Automated Program Repaircited by 37
  32. 2023Joint Prompt Optimization of Stacked LLMs using Variational Inferencecited by 32
  33. 2024What Makes Large Language Models Reason in (Multi-Turn) Code Generation?cited by 29
  34. 2024CodeS: Natural Language to Code Repository via Multi-Layer Sketchcited by 28
  35. 2024The Importance of Directional Feedback for LLM-based Optimizerscited by 26
  36. 2021A Systematic Survey of Text Worlds as Embodied Natural Language Environmentscited by 17
  37. 2024Natural Language Reinforcement Learningcited by 17
  38. 2024Policy Improvement using Language Feedback Modelscited by 16

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 8; 2026: 11.

19 papers.

Sort:

By area:

  1. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 195
  2. 2025BugPilot: Complex Bug Generation for Efficient Learning of SWE Skillscited by 28
  3. 2025MLRC-Bench: Can Language Agents Solve Machine Learning Research Challenges?cited by 27
  4. 2025From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligencecited by 13
  5. 2026SciAgentGym: Benchmarking Multi-Step Scientific Tool-use in LLM Agentscited by 12
  6. 2026AgentStepper: Interactive Debugging of Software Development Agentscited by 8
  7. 2025TextAtari: 100K Frames Game Playing with Language Agentscited by 6
  8. 2026Leyline: KV Cache Directives for Agentic Inferencecited by 5
  9. 2025kAgent: An execution-guided crash resolution agent for the Linux kernelcited by 4
  10. 2025Gistify! Codebase-Level Understanding via Runtime Executioncited by 4
  11. 2026FrogNano: Training a 4B Coding Agent via Online Task Synthesiscited by 3
  12. 2025TENET: One Step Toward Test-Driven Development for Repository-Level Code Generationcited by 2
7 more
  1. 2026ScienceIDE: Turning World's Scientific Codebase into Agent Learnable Environmentscited by 2
  2. 2026Debug2Fix: Can Interactive Debugging Help Coding Agents Fix More Bugs?cited by 1
  3. 2026Precise Debugging Benchmark: Is Your Model Debugging or Regenerating?cited by 1
  4. 2026Empowering Autonomous Debugging Agents with Efficient Dynamic Analysiscited by 1
  5. 2026Harbor Adapters and Harbor-Index: Infrastructure and a Curated Meta-Dataset for Large-Scale Agentic Evaluationcited by 1
  6. 2026Evaluating RL Explainability Methods by How Much They Help Fix Bugs in Agentscited by 0
  7. 2026Terminal Agents: A Survey of AI Agents in Command-Line Environmentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024COAST: Enhancing the Code Debugging Ability of LLMs through Communicative Agent Based Data Synthesis
  2. 2024RGD: Multi-LLM Based Agent Debugger via Refinement and Generation Guidance
  3. 2025Enhancing LLM Code Generation: A Systematic Evaluation of Multi-Agent Collaboration and Runtime Debugging for Improved Accuracy, Reliability, and Latency
  4. 2025DebugTA: An LLM-Based Agent for Simplifying Debugging and Teaching in Programming Education
  5. 2025DoVer: Intervention-Driven Auto Debugging for LLM Multi-Agent Systems
  6. 2025Fully Autonomous Programming using Iterative Multi-Agent Debugging with Large Language Models
  7. 2025Illuminating LLM Coding Agents: Visual Analytics for Deeper Understanding and Enhancement
  8. 2025Towards Adaptive Software Agents for Debugging