# Pre-training Code Representation with Semantic Flow Graph for Effective Bug Localization

arXiv: https://arxiv.org/abs/2308.12773 · HTML: https://mlsanity.com/paper/2308.12773/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Aug 2023.

## Built on by (most cited first, 8 of 8)

- [Large Language Models for Cyber Security: A Systematic Literature Review](https://mlsanity.com/paper/2405.04760/) (2024)
- [BLAZE: Cross-Language and Cross-Project Bug Localization via Dynamic Chunking and Hard Example Learning](https://mlsanity.com/paper/2407.17631/) (2024)
- [ObscuraCoder: Powering Efficient Code LM Pre-Training Via Obfuscation Grounding](https://mlsanity.com/paper/2504.00019/) (2025)
- [Post-Incorporating Code Structural Knowledge into Pretrained Models via ICL for Code Translation](https://mlsanity.com/paper/2503.22776/) (2025)
- [NARRepair: Non-Autoregressive Code Generation Model for Automatic Program Repair](https://mlsanity.com/paper/2406.16526/) (2024)
- [Large Language Models in Cybersecurity: Applications, Vulnerabilities, and Defense Techniques](https://mlsanity.com/paper/2507.13629/) (2025)
- [Design-Specification Tiling for ICL-based CAD Code Generation](https://mlsanity.com/paper/2603.12712/) (2026)
- [CIT-CAD: Constraint Intent Tree-based CAD Code Generation and Verification](https://mlsanity.com/paper/2609.07434/) (2026)

## Abstract (arXiv)

Enlightened by the big success of pre-training in natural language processing, pre-trained models for programming languages have been widely used to promote code intelligence in recent years. In particular, BERT has been used for bug localization tasks and impressive results have been obtained. However, these BERT-based bug localization techniques suffer from two issues. First, the pre-trained BERT model on source code does not adequately capture the deep semantics of program code. Second, the overall bug localization models neglect the necessity of large-scale negative samples in contrastive learning for representations of changesets and ignore the lexical similarity between bug reports and changesets during similarity estimation. We address these two issues by 1) proposing a novel directed, multiple-label code graph representation named Semantic Flow Graph (SFG), which compactly and adequately captures code semantics, 2) designing and training SemanticCodeBERT based on SFG, and 3) designing a novel Hierarchical Momentum Contrastive Bug Localization technique (HMCBL). Evaluation results show that our method achieves state-of-the-art performance in bug localization.