# Self-Supervised Bug Detection and Repair

arXiv: https://arxiv.org/abs/2105.12787 · HTML: https://mlsanity.com/paper/2105.12787/

## Key facts

- Cited by 59 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 34 references that are in the map.
- First posted on arXiv 26 May 2021.

## Builds on (most cited first, 17 of 17)

- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [Gated Graph Sequence Neural Networks](https://mlsanity.com/paper/1511.05493/) (2015)
- [Going deeper with Image Transformers](https://mlsanity.com/paper/2103.17239/) (2021)
- [Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Design](https://mlsanity.com/paper/2012.02096/) (2020)
- [ReZero is All You Need: Fast Convergence at Large Depth](https://mlsanity.com/paper/2003.04887/) (2020)
- [A Survey of Machine Learning for Big Code and Naturalness](https://mlsanity.com/paper/1709.06182/) (2017)
- [Learning to Represent Programs with Graphs](https://mlsanity.com/paper/1711.00740/) (2017)
- [The Adverse Effects of Code Duplication in Machine Learning Models of Code](https://mlsanity.com/paper/1812.06469/) (2018)
- [DeepBugs: A Learning Approach to Name-based Bug Detection](https://mlsanity.com/paper/1805.11683/) (2018)
- [Semantic Robustness of Models of Source Code](https://mlsanity.com/paper/2002.03043/) (2020)
- [LambdaNet: Probabilistic Type Inference using Graph Neural Networks](https://mlsanity.com/paper/2005.02161/) (2020)
- [Neural Program Repair by Jointly Learning to Localize and Repair](https://mlsanity.com/paper/1904.01720/) (2019)
- [How Often Do Single-Statement Bugs Occur? The ManySStuBs4J Dataset](https://mlsanity.com/paper/1905.13334/) (2019)
- [On the "Naturalness" of Buggy Code](https://mlsanity.com/paper/1506.01159/) (2015)
- [Typilus: Neural Type Hints](https://mlsanity.com/paper/2004.10657/) (2020)
- [Learning to Fix Build Errors with Graph2Diff Neural Networks](https://mlsanity.com/paper/1911.01205/) (2019)

## Built on by (most cited first, 30 of 59)

- [An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generation](https://mlsanity.com/paper/2302.06527/) (2023)
- [CodeBERTScore: Evaluating Code Generation with Pretrained Models of Code](https://mlsanity.com/paper/2302.05527/) (2023)
- [RAP-Gen: Retrieval-Augmented Patch Generation with CodeT5 for Automatic Program Repair](https://mlsanity.com/paper/2309.06057/) (2023)
- [Natural Language Generation and Understanding of Big Code for AI-Assisted Programming: A Review](https://mlsanity.com/paper/2307.02503/) (2023)
- [Large Language Models of Code Fail at Completing Code with Potential Bugs](https://mlsanity.com/paper/2306.03438/) (2023)
- [Multi-target Backdoor Attacks for Code Pre-trained Models](https://mlsanity.com/paper/2306.08350/) (2023)
- [Evaluating LLMs' Mathematical and Coding Competency through Ontology-guided Interventions](https://mlsanity.com/paper/2401.09395/) (2024)
- [Towards Learning (Dis)-Similarity of Source Code from Program Contrasts](https://mlsanity.com/paper/2110.03868/) (2021)
- [ITER: Iterative Neural Repair for Multi-Location Patches](https://mlsanity.com/paper/2304.12015/) (2023)
- [MdEval: Massively Multilingual Code Debugging](https://mlsanity.com/paper/2411.02310/) (2024)
- [On Distribution Shift in Learning-based Bug Detectors](https://mlsanity.com/paper/2204.10049/) (2022)
- [HEAT: Hyperedge Attention Networks](https://mlsanity.com/paper/2201.12113/) (2022)
- [Static Prediction of Runtime Errors by Learning to Execute Programs with External Resource Descriptions](https://mlsanity.com/paper/2203.03771/) (2022)
- [RunBugRun -- An Executable Dataset for Automated Program Repair](https://mlsanity.com/paper/2304.01102/) (2023)
- [Detect-Localize-Repair: A Unified Framework for Learning to Debug with CodeT5](https://mlsanity.com/paper/2211.14875/) (2022)
- [Improving Automated Program Repair with Domain Adaptation](https://mlsanity.com/paper/2212.11414/) (2022)
- [Socratic-SWE: Self-Evolving Coding Agents via Trace-Derived Agent Skills](https://mlsanity.com/paper/2606.07412/) (2026)
- [Fix Bugs with Transformer through a Neural-Symbolic Edit Grammar](https://mlsanity.com/paper/2204.06643/) (2022)
- [DeepCode AI Fix: Fixing Security Vulnerabilities with Large Language Models](https://mlsanity.com/paper/2402.13291/) (2024)
- [Are Large Language Models Robust in Understanding Code Against Semantics-Preserving Mutations?](https://mlsanity.com/paper/2505.10443/) (2025)
- [Mitigating Data Imbalance for Software Vulnerability Assessment: Does Data Augmentation Help?](https://mlsanity.com/paper/2407.10722/) (2024)
- [RAGVA: Engineering Retrieval Augmented Generation-based Virtual Assistants in Practice](https://mlsanity.com/paper/2502.14930/) (2025)
- [Boosting Source Code Learning with Text-Oriented Data Augmentation: An Empirical Study](https://mlsanity.com/paper/2303.06808/) (2023)
- [Learning Defect Prediction from Unrealistic Data](https://mlsanity.com/paper/2311.00931/) (2023)
- [IFEvalCode: Controlled Code Generation](https://mlsanity.com/paper/2507.22462/) (2025)
- [A Survey of Deep Learning Models for Structural Code Understanding](https://mlsanity.com/paper/2205.01293/) (2022)
- [Can we learn from developer mistakes? Learning to localize and repair real bugs from real bug fixes](https://mlsanity.com/paper/2207.00301/) (2022)
- [A Library for Representing Python Programs as Graphs for Machine Learning](https://mlsanity.com/paper/2208.07461/) (2022)
- [MIXCODE: Enhancing Code Classification by Mixup-Based Data Augmentation](https://mlsanity.com/paper/2210.03003/) (2022)
- [Self-Supervised Learning to Prove Equivalence Between Straight-Line Programs via Rewrite Rules](https://mlsanity.com/paper/2109.10476/) (2021)

## Abstract (arXiv)

Machine learning-based program analyses have recently shown the promise of integrating formal and probabilistic reasoning towards aiding software development. However, in the absence of large annotated corpora, training these analyses is challenging. Towards addressing this, we present BugLab, an approach for self-supervised learning of bug detection and repair. BugLab co-trains two models: (1) a detector model that learns to detect and repair bugs in code, (2) a selector model that learns to create buggy code for the detector to use as training data. A Python implementation of BugLab improves by up to 30% upon baseline methods on a test dataset of 2374 real-life bugs and finds 19 previously unknown bugs in open-source software.