mlsanityData to 14 Oct 2026

Imitation Game: Reproducing Deep Learning Bugs Leveraging an Intelligent Agent

arXiv 2512.14990 ↗First posted 17 Dec 2025cs.SE cs.AI cs.LGcited by 1

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

Despite their wide adoption in various domains (e.g., healthcare, finance, software engineering), Deep Learning (DL)-based applications suffer from many bugs, failures, and vulnerabilities. Reproducing these bugs is essential for their resolution, but it is extremely challenging due to the inherent nondeterminism of DL models and their tight coupling with hardware and software environments. According to recent studies, only about 3% of DL bugs can be reliably reproduced using manual approaches. To address these challenges, we present RepGen, a novel, automated, and intelligent approach for reproducing deep learning bugs. RepGen constructs a learning-enhanced context from a project, develops a comprehensive plan for bug reproduction, employs an iterative generate-validate-refine mechanism, and thus generates such code using an LLM that reproduces the bug at hand. We evaluate RepGen on 106 real-world deep learning bugs and achieve a reproduction rate of 80.19%, a 19.81% improvement over the state-of-the-art measure. A developer study involving 27 participants shows that RepGen improves the success rate of DL bug reproduction by 23.35%, reduces the time to reproduce by 56.8%, and lowers participants' cognitive load.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 33 of its 82 references that are in the map. First posted on arXiv 17 Dec 2025.

Builds on

33 of 82 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024RepairAgent: An Autonomous, LLM-Based Agent for Program Repair25 Mar

Other references · 32

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  4. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  5. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
  6. 2019A Survey of Deep Learning Techniques for Autonomous Drivingcited by 403
  7. 2023Self-planning Code Generation with Large Language Modelscited by 124
  8. 2022Large Language Models are Few-shot Testers: Exploring LLM-based General Bug Reproductioncited by 111
  9. 2019Taxonomy of Real Faults in Deep Learning Systemscited by 90
  10. 2019A Comprehensive Study on Deep Learning Bug Characteristicscited by 65
  11. 2022Compilable Neural Code Generation with Compiler Feedbackcited by 47
  12. 2024Reasoning Runtime Behavior of a Program with LLM: How Far Are We?cited by 45
20 more
  1. 2023Prompting Is All You Need: Automated Android Bug Replay with Large Language Modelscited by 40
  2. 2021DeepLocalize: Fault Localization for Deep Neural Networkscited by 38
  3. 2021What do pre-trained code models know about code?cited by 36
  4. 2021Bridging Pre-trained Models and Downstream Tasks for Source Code Understandingcited by 28
  5. 2021Silent Bugs in Deep Learning Frameworks: An Empirical Study of Keras and TensorFlowcited by 25
  6. 2021An Empirical Study on the Usage of BERT Models for Code Completioncited by 22
  7. 2022DEAR: A Novel Deep Learning-based Approach for Automated Program Repaircited by 22
  8. 2022DeepFD: Automated Fault Diagnosis and Localization for Deep Learning Programscited by 16
  9. 2022Bugs in Machine Learning-based Systems: A Faultload Benchmarkcited by 15
  10. 2023Bug Characterization in Machine Learning-based Systemscited by 10
  11. 2022Demystifying Dependency Bugs in Deep Learning Stackcited by 9
  12. 2023RLocator: Reinforcement Learning for Bug Localizationcited by 7
  13. 2024Feedback-Driven Automated Whole Bug Report Reproduction for Android Appscited by 7
  14. 2024Towards Enhancing the Reproducibility of Deep Learning Bugs: An Empirical Studycited by 6
  15. 2024Towards Understanding the Impact of Data Bugs on Deep Learning Models in Software Engineeringcited by 6
  16. 2017Evolution of statistical analysis in empirical software engineering research: Current state and steps forwardcited by 5
  17. 2023On Using GUI Interaction Data to Improve Text Retrieval-based Bug Localizationcited by 5
  18. 2025Improved IR-based Bug Localization with Intelligent Relevance Feedbackcited by 5
  19. 2023GIRT-Data: Sampling GitHub Issue Report Templatescited by 2
  20. 2024Checker Bug Detection and Repair in Deep Learning Librariescited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Reproducible, Explainable, and Effective Evaluations of Agentic AI for Software Engineeringcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026ReproRepo: Scaling Reproducibility Audits with GitHub Repository Issues
  2. 2025One Tool Is Enough: Reinforcement Learning for Repository-Level LLM Agents
  3. 2025RepoForge: Training a SOTA Fast-thinking SWE Agent with an End-to-End Data Curation Pipeline Synergizing SFT and RL at Scale
  4. 2026RepoLaunch: Automating Build and Management of Code Repositories across Languages and Platforms
  5. 2026EvolveCoder: Evolving Test Cases via Adversarial Verification for Code Reinforcement Learning
  6. 2025ReCode: Updating Code API Knowledge with Reinforcement Learning
  7. 2026Automated Modernization of Machine Learning Engineering Notebooks for Reproducibility
  8. 2026Rollout Cards: A Reproducibility Standard for Agent Research