# CLNX: Bridging Code and Natural Language for C/C++ Vulnerability-Contributing Commits Identification

arXiv: https://arxiv.org/abs/2409.07407 · HTML: https://mlsanity.com/paper/2409.07407/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 40 references that are in the map.
- First posted on arXiv 11 Sep 2024.

## Builds on (most cited first, 22 of 22)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [CodeBERT: A Pre-Trained Model for Programming and Natural Languages](https://mlsanity.com/paper/2002.08155/) (2020)
- [CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation](https://mlsanity.com/paper/2102.04664/) (2021)
- [A Systematic Evaluation of Large Language Models of Code](https://mlsanity.com/paper/2202.13169/) (2022)
- [Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networks](https://mlsanity.com/paper/1909.03496/) (2019)
- [Deep Learning based Vulnerability Detection: Are We There Yet?](https://mlsanity.com/paper/2009.07235/) (2020)
- [Automated Vulnerability Detection in Source Code Using Deep Representation Learning](https://mlsanity.com/paper/1807.04320/) (2018)
- [SySeVR: A Framework for Using Deep Learning to Detect Software Vulnerabilities](https://mlsanity.com/paper/1807.06756/) (2018)
- [VulBERTa: Simplified Source Code Pre-Training for Vulnerability Detection](https://mlsanity.com/paper/2205.12424/) (2022)
- [An Empirical Study of Deep Learning Models for Vulnerability Detection](https://mlsanity.com/paper/2212.08109/) (2022)
- [How fine can fine-tuning be? Learning efficient language models](https://mlsanity.com/paper/2004.14129/) (2020)
- [Transformer-Based Language Models for Software Vulnerability Detection](https://mlsanity.com/paper/2204.03214/) (2022)
- [VulDeeLocator: A Deep Learning-based Fine-grained Vulnerability Detector](https://mlsanity.com/paper/2001.02350/) (2020)
- [VELVET: a noVel Ensemble Learning approach to automatically locate VulnErable sTatements](https://mlsanity.com/paper/2112.10893/) (2021)
- [ContraBERT: Enhancing Code Pre-trained Models via Contrastive Learning](https://mlsanity.com/paper/2301.09072/) (2023)
- [MVD: Memory-Related Vulnerability Detection Based on Flow-Sensitive Graph Neural Networks](https://mlsanity.com/paper/2203.02660/) (2022)
- [On using distributed representations of source code for the detection of C security vulnerabilities](https://mlsanity.com/paper/2106.01367/) (2021)
- [Enhancing Security Patch Identification by Capturing Structures in Commits](https://mlsanity.com/paper/2207.09022/) (2022)

## Abstract (arXiv)

Large Language Models (LLMs) have shown great promise in vulnerability identification. As C/C++ comprises half of the Open-Source Software (OSS) vulnerabilities over the past decade and updates in OSS mainly occur through commits, enhancing LLMs' ability to identify C/C++ Vulnerability-Contributing Commits (VCCs) is essential. However, current studies primarily focus on further pre-training LLMs on massive code datasets, which is resource-intensive and poses efficiency challenges. In this paper, we enhance the ability of BERT-based LLMs to identify C/C++ VCCs in a lightweight manner. We propose CodeLinguaNexus (CLNX) as a bridge facilitating communication between C/C++ programs and LLMs. Based on commits, CLNX efficiently converts the source code into a more natural representation while preserving key details. Specifically, CLNX first applies structure-level naturalization to decompose complex programs, followed by token-level naturalization to interpret complex symbols. We evaluate CLNX on public datasets of 25,872 C/C++ functions with their commits. The results show that CLNX significantly enhances the performance of LLMs on identifying C/C++ VCCs. Moreover, CLNX-equipped CodeBERT achieves new state-of-the-art and identifies 38 OSS vulnerabilities in the real world.