# Vul-LMGNNs: Fusing language models and online-distilled graph neural networks for code vulnerability detection

arXiv: https://arxiv.org/abs/2404.14719 · HTML: https://mlsanity.com/paper/2404.14719/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Apr 2024.

## Built on by (most cited first, 11 of 11)

- [Ethereum Fraud Detection via Joint Transaction Language Model and Graph Representation Learning](https://mlsanity.com/paper/2409.07494/) (2024)
- [A Systematic Literature Review on Detecting Software Vulnerabilities with Large Language Models](https://mlsanity.com/paper/2507.22659/) (2025)
- [Automated Software Vulnerability Static Code Analysis Using Generative Pre-Trained Transformer Models](https://mlsanity.com/paper/2408.00197/) (2024)
- [From Past to Present: A Survey of Malicious URL Detection Techniques, Datasets and Code Repositories](https://mlsanity.com/paper/2504.16449/) (2025)
- [Breaking Obfuscation: Cluster-Aware Graph with LLM-Aided Recovery for Malicious JavaScript Detection](https://mlsanity.com/paper/2507.22447/) (2025)
- [Transducer Tuning: Efficient Model Adaptation for Software Tasks Using Code Property Graphs](https://mlsanity.com/paper/2412.13467/) (2024)
- [Transformers in Protein: A Survey](https://mlsanity.com/paper/2505.20098/) (2025)
- [GraphTorque: Torque-Driven Rewiring Graph Neural Network](https://mlsanity.com/paper/2507.21422/) (2025)
- [Focus on What Matters: Fisher-Guided Adaptive Multimodal Fusion for Vulnerability Detection](https://mlsanity.com/paper/2601.02438/) (2026)
- [Program Structure-aware Language Models: Targeted Software Testing beyond Textual Semantics](https://mlsanity.com/paper/2604.17715/) (2026)
- [PLMGH: What Matters in PLM-GNN Hybrids for Code Classification and Vulnerability Detection](https://mlsanity.com/paper/2604.25599/) (2026)

## Abstract (arXiv)

Code Language Models (codeLMs) and Graph Neural Networks (GNNs) are widely used in code vulnerability detection. However, GNNs often rely on aggregating information from adjacent nodes, limiting structural information propagation across layers. While codeLMs can supplement GNNs with semantic information, existing integration methods underexplore their collaborative potential. To address these challenges, we propose Vul-LMGNNs, integrating pre-trained codeLMs with GNNs to enable cross-layer propagation of semantic and structural information. Vul-LMGNNs leverage Code Property Graphs (CPGs) to incorporate syntax, control flow, and data dependencies, using gated GNNs for structural extraction. An online knowledge distillation (KD) mechanism allows a student GNN to capture structural information from a trained counterpart via alternating training. Additionally, an "implicit-explicit" joint training framework leverages codeLMs to initialize embeddings and propagate code semantics. In the explicit phase, it performs late fusion via linear interpolation. Evaluations on real-world vulnerability datasets show Vul-LMGNNs outperform 17 state-of-the-art approaches. Source code is available at: https://github.com/Vul-LMGNN/vul-LMGNN.