# Enhancing Reverse Engineering: Investigating and Benchmarking Large Language Models for Vulnerability Analysis in Decompiled Binaries

arXiv: https://arxiv.org/abs/2411.04981 · HTML: https://mlsanity.com/paper/2411.04981/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 30 references that are in the map.
- First posted on arXiv 7 Nov 2024.

## Builds on (most cited first, 9 of 9)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [StarCoder: may the source be with you!](https://mlsanity.com/paper/2305.06161/) (2023)
- [CodeGen2: Lessons for Training LLMs on Programming and Natural Languages](https://mlsanity.com/paper/2305.02309/) (2023)
- [Source Code Foundation Models are Transferable Binary Analysis Knowledge Bases](https://mlsanity.com/paper/2405.19581/) (2024)
- [Asteria-Pro: Enhancing Deep-Learning Based Binary Code Similarity Detection by Incorporating Domain Knowledge](https://mlsanity.com/paper/2301.00511/) (2023)
- [Binary-level Directed Fuzzing for Use-After-Free Vulnerabilities](https://mlsanity.com/paper/2002.10751/) (2020)

## Built on by (most cited first, 6 of 6)

- [CREBench: Evaluating Large Language Models in Cryptographic Binary Reverse Engineering](https://mlsanity.com/paper/2604.03750/) (2026)
- [Large Language Models in Cybersecurity: Applications, Vulnerabilities, and Defense Techniques](https://mlsanity.com/paper/2507.13629/) (2025)
- [NeuroDeX: Unlocking Diverse Support in Decompiling Deep Neural Network Executables](https://mlsanity.com/paper/2509.06402/) (2025)
- [Analyzing Chain of Thought (CoT) Approaches in Control Flow Code Deobfuscation Tasks](https://mlsanity.com/paper/2604.15390/) (2026)
- [Multi-View Decompilation for LLM-Based Malware Classification](https://mlsanity.com/paper/2606.20436/) (2026)
- [FuzzingBrain-Bench V1: Evaluating Open-Ended Bug Discovery by LLMs](https://mlsanity.com/paper/2608.25158/) (2026)

## Abstract (arXiv)

Security experts reverse engineer (decompile) binary code to identify critical security vulnerabilities. The limited access to source code in vital systems - such as firmware, drivers, and proprietary software used in Critical Infrastructures (CI) - makes this analysis even more crucial on the binary level. Even with available source code, a semantic gap persists after compilation between the source and the binary code executed by the processor. This gap may hinder the detection of vulnerabilities in source code. That being said, current research on Large Language Models (LLMs) overlooks the significance of decompiled binaries in this area by focusing solely on source code. In this work, we are the first to empirically uncover the substantial semantic limitations of state-of-the-art LLMs when it comes to analyzing vulnerabilities in decompiled binaries, largely due to the absence of relevant datasets. To bridge the gap, we introduce DeBinVul, a novel decompiled binary code vulnerability dataset. Our dataset is multi-architecture and multi-optimization, focusing on C/C++ due to their wide usage in CI and association with numerous vulnerabilities. Specifically, we curate 150,872 samples of vulnerable and non-vulnerable decompiled binary code for the task of (i) identifying; (ii) classifying; (iii) describing vulnerabilities; and (iv) recovering function names in the domain of decompiled binaries. Subsequently, we fine-tune state-of-the-art LLMs using DeBinVul and report on a performance increase of 19%, 24%, and 21% in the capabilities of CodeLlama, Llama3, and CodeGen2 respectively, in detecting binary code vulnerabilities. Additionally, using DeBinVul, we report a high performance of 80-90% on the vulnerability classification task. Furthermore, we report improved performance in function name recovery and vulnerability description tasks.