# DSHGT: Dual-Supervisors Heterogeneous Graph Transformer -- A pioneer study of using heterogeneous graph learning for detecting software vulnerabilities

arXiv: https://arxiv.org/abs/2306.01376 · HTML: https://mlsanity.com/paper/2306.01376/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 56 references that are in the map.
- First posted on arXiv 2 Jun 2023.

## Builds on (most cited first, 23 of 23)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [CodeBERT: A Pre-Trained Model for Programming and Natural Languages](https://mlsanity.com/paper/2002.08155/) (2020)
- [Gated Graph Sequence Neural Networks](https://mlsanity.com/paper/1511.05493/) (2015)
- [Distributed Representations of Sentences and Documents](https://mlsanity.com/paper/1405.4053/) (2014)
- [GraphCodeBERT: Pre-training Code Representations with Data Flow](https://mlsanity.com/paper/2009.08366/) (2020)
- [Heterogeneous Graph Transformer](https://mlsanity.com/paper/2003.01332/) (2020)
- [Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks from the Topological View](https://mlsanity.com/paper/1909.03211/) (2019)
- [Unified Pre-training for Program Understanding and Generation](https://mlsanity.com/paper/2103.06333/) (2021)
- [Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networks](https://mlsanity.com/paper/1909.03496/) (2019)
- [Learning to Represent Programs with Graphs](https://mlsanity.com/paper/1711.00740/) (2017)
- [AMMUS : A Survey of Transformer-based Pretrained Models in Natural Language Processing](https://mlsanity.com/paper/2108.05542/) (2021)
- [A Survey on Heterogeneous Graph Embedding: Methods, Techniques, Applications and Sources](https://mlsanity.com/paper/2011.14867/) (2020)
- [SySeVR: A Framework for Using Deep Learning to Detect Software Vulnerabilities](https://mlsanity.com/paper/1807.06756/) (2018)
- [DeepBugs: A Learning Approach to Name-based Bug Detection](https://mlsanity.com/paper/1805.11683/) (2018)
- [Getafix: Learning to Fix Bugs Automatically](https://mlsanity.com/paper/1902.06111/) (2019)
- [Multi-Modal Attention Network Learning for Semantic Source Code Retrieval](https://mlsanity.com/paper/1909.13516/) (2019)
- [VulDeeLocator: A Deep Learning-based Fine-grained Vulnerability Detector](https://mlsanity.com/paper/2001.02350/) (2020)
- [An Empirical Study on the Usage of Transformer Models for Code Completion](https://mlsanity.com/paper/2108.01585/) (2021)
- [Vulnerability Detection with Graph Simplification and Enhanced Graph Representation Learning](https://mlsanity.com/paper/2302.04675/) (2023)
- [GPS: A Policy-driven Sampling Approach for Graph Representation Learning](https://mlsanity.com/paper/2112.14482/) (2021)
- [AstBERT: Enabling Language Model for Financial Code Understanding with Abstract Syntax Trees](https://mlsanity.com/paper/2201.07984/) (2022)

## Built on by (most cited first, 2 of 2)

- [A Structure-Agnostic Co-Tuning Framework for LLMs and SLMs in Cloud-Edge Systems](https://mlsanity.com/paper/2511.11678/) (2025)
- [Software Vulnerability Detection Using a Lightweight Graph Neural Network](https://mlsanity.com/paper/2603.29216/) (2026)

## Abstract (arXiv)

Vulnerability detection is a critical problem in software security and attracts growing attention both from academia and industry. Traditionally, software security is safeguarded by designated rule-based detectors that heavily rely on empirical expertise, requiring tremendous effort from software experts to generate rule repositories for large code corpus. Recent advances in deep learning, especially Graph Neural Networks (GNN), have uncovered the feasibility of automatic detection of a wide range of software vulnerabilities. However, prior learning-based works only break programs down into a sequence of word tokens for extracting contextual features of codes, or apply GNN largely on homogeneous graph representation (e.g., AST) without discerning complex types of underlying program entities (e.g., methods, variables). In this work, we are one of the first to explore heterogeneous graph representation in the form of Code Property Graph and adapt a well-known heterogeneous graph network with a dual-supervisor structure for the corresponding graph learning task. Using the prototype built, we have conducted extensive experiments on both synthetic datasets and real-world projects. Compared with the state-of-the-art baselines, the results demonstrate promising effectiveness in this research direction in terms of vulnerability detection performance (average F1 improvements over 10% in real-world projects) and transferability from C/C++ to other programming languages (average F1 improvements over 11%).