mlsanityData to 7 Oct 2026

GitBug-Java: A Reproducible Benchmark of Recent Java Bugs

arXiv 2402.02961 ↗First posted 5 Feb 2024cs.SEcited by 13

Not on the map. Why?

Abstract (arXiv)

Bug-fix benchmarks are essential for evaluating methodologies in automatic program repair (APR) and fault localization (FL). However, existing benchmarks, exemplified by Defects4J, need to evolve to incorporate recent bug-fixes aligned with contemporary development practices. Moreover, reproducibility, a key scientific principle, has been lacking in bug-fix benchmarks. To address these gaps, we present GitBug-Java, a reproducible benchmark of recent Java bugs. GitBug-Java features 199 bugs extracted from the 2023 commit history of 55 notable open-source repositories. The methodology for building GitBug-Java ensures the preservation of bug-fixes in fully-reproducible environments. We publish GitBug-Java at https://github.com/gitbugactions/gitbug-java.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Feb 2024.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 5; 2026: 4.

13 papers.

Sort:

By area:

  1. 2024RepairAgent: An Autonomous, LLM-Based Agent for Program Repaircited by 225
  2. 2024You Name It, I Run It: An LLM Agent to Execute Tests of Arbitrary Projectscited by 45
  3. 2025LessLeak-Bench: A First Investigation of Data Leakage in LLMs Across 83 Software Engineering Benchmarkscited by 34
  4. 2024Are Large Language Models Memorizing Bug Benchmarks?cited by 11
  5. 2024RepairBench: Leaderboard of Frontier Models for Program Repaircited by 9
  6. 2025Improving Deep Assertion Generation via Fine-Tuning Retrieval-Augmented Pre-trained Language Modelscited by 4
  7. 2025Benchmarking AI Models in Software Engineering: A Review, Search Tool, and Unified Approach for Elevating Benchmark Qualitycited by 3
  8. 2025Improving Retrieval-Augmented Deep Assertion Generation via Joint Trainingcited by 2
  9. 2025Are Sparse Autoencoders Useful for Java Function Bug Detection?cited by 2
  10. 2026Debug2Fix: Can Interactive Debugging Help Coding Agents Fix More Bugs?cited by 1
  11. 2026A Metamorphic Testing Approach to Diagnosing Memorization in LLM-Based Program Repaircited by 0
  12. 2026Do Coverage and Mutation Scores of LLM-Generated Test Suites Correlate with Their Effectiveness? (Replicability Study)cited by 0
1 more
  1. 2026PatchBench: Evaluating AI Agents for Vulnerability Patchingcited by 0