# A Systematic Study of Code Obfuscation Against LLM-based Vulnerability Detection

arXiv: https://arxiv.org/abs/2512.16538 · HTML: https://mlsanity.com/paper/2512.16538/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 80 references that are in the map.
- First posted on arXiv 18 Dec 2025.

## Builds on (most cited first, 30 of 38)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [CodeBERT: A Pre-Trained Model for Programming and Natural Languages](https://mlsanity.com/paper/2002.08155/) (2020)
- [Qwen2.5-Coder Technical Report](https://mlsanity.com/paper/2409.12186/) (2024)
- [GraphCodeBERT: Pre-training Code Representations with Data Flow](https://mlsanity.com/paper/2009.08366/) (2020)
- [Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions](https://mlsanity.com/paper/2108.09293/) (2021)
- [Vulnerability Detection with Code Language Models: How Far Are We?](https://mlsanity.com/paper/2403.18624/) (2024)
- [DiverseVul: A New Vulnerable Source Code Dataset for Deep Learning Based Vulnerability Detection](https://mlsanity.com/paper/2304.00409/) (2023)
- [Natural Attack for Pre-trained Models of Code](https://mlsanity.com/paper/2201.08698/) (2022)
- [Vul-RAG: Enhancing LLM-based Vulnerability Detection via Knowledge-level RAG](https://mlsanity.com/paper/2406.11147/) (2024)
- [Adversarial Examples for Models of Code](https://mlsanity.com/paper/1910.07517/) (2019)
- [Understanding the Effectiveness of Large Language Models in Detecting Security Vulnerabilities](https://mlsanity.com/paper/2311.16169/) (2023)
- [CodeAttack: Code-Based Adversarial Attacks for Pre-trained Programming Language Models](https://mlsanity.com/paper/2206.00052/) (2022)
- [VulBERTa: Simplified Source Code Pre-Training for Vulnerability Detection](https://mlsanity.com/paper/2205.12424/) (2022)
- [Data Quality for Software Vulnerability Datasets](https://mlsanity.com/paper/2301.05456/) (2023)
- [Prompt-Enhanced Software Vulnerability Detection Using ChatGPT](https://mlsanity.com/paper/2308.12697/) (2023)
- [An Empirical Study of Deep Learning Models for Vulnerability Detection](https://mlsanity.com/paper/2212.08109/) (2022)
- [Large Language Model for Vulnerability Detection: Emerging Results and Future Directions](https://mlsanity.com/paper/2401.15468/) (2024)
- [ChatGPT for Vulnerability Detection, Classification, and Repair: How Far Are We?](https://mlsanity.com/paper/2310.09810/) (2023)
- [An LLM-Assisted Easy-to-Trigger Backdoor Attack on Code Completion Models: Injecting Disguised Vulnerabilities against Strong Detection](https://mlsanity.com/paper/2406.06822/) (2024)
- [Transformer-Based Language Models for Software Vulnerability Detection](https://mlsanity.com/paper/2204.03214/) (2022)
- [Empirical Review of Automated Analysis Tools on 47,587 Ethereum Smart Contracts](https://mlsanity.com/paper/1910.10601/) (2019)
- [Uncovering the Limits of Machine Learning for Automatic Vulnerability Detection](https://mlsanity.com/paper/2306.17193/) (2023)
- [VulEval: Towards Repository-Level Evaluation of Software Vulnerability Detection](https://mlsanity.com/paper/2404.15596/) (2024)
- [Comparison of Static Application Security Testing Tools and Large Language Models for Repo-level Vulnerability Detection](https://mlsanity.com/paper/2407.16235/) (2024)
- [AIBugHunter: A Practical Tool for Predicting, Classifying and Repairing Software Vulnerabilities](https://mlsanity.com/paper/2305.16615/) (2023)
- [Pre-training by Predicting Program Dependencies for Vulnerability Analysis Tasks](https://mlsanity.com/paper/2402.00657/) (2024)
- [Distinguishing Look-Alike Innocent and Vulnerable Code by Subtle Semantic Representation Learning and Explanation](https://mlsanity.com/paper/2308.11237/) (2023)

## Built on by (most cited first, 5 of 5)

- [Detecting Data Poisoning in Code Generation LLMs via Black-Box, Vulnerability-Oriented Scanning](https://mlsanity.com/paper/2603.17174/) (2026)
- [Poisoned Identifiers Survive LLM Deobfuscation: A Case Study on Claude Opus 4.6](https://mlsanity.com/paper/2604.04289/) (2026)
- [Probing the Prefill: Detecting Code Vulnerabilities via Latent Activations](https://mlsanity.com/paper/2608.16970/) (2026)
- [Cheap to Hypothesize, Costly to Verify: The Defense Surface of Agentic Vulnerability Discovery](https://mlsanity.com/paper/2609.35909/) (2026)
- [Natural Code Transformation Confuses Agents in Vulnerability Analysis](https://mlsanity.com/paper/2610.14350/) (2026)

## Abstract (arXiv)

As large language models (LLMs) are increasingly adopted for code vulnerability detection, their reliability and robustness across diverse vulnerability types have become a pressing concern. In traditional adversarial settings, code obfuscation has long been used as a general strategy to bypass auditing tools, preserving exploitability without tampering with the tools themselves. Numerous efforts have explored obfuscation methods and tools, yet their capabilities differ in terms of supported techniques, granularity, and programming languages, making it difficult to systematically assess their impact on LLM-based vulnerability detection. To address this gap, we provide a structured systematization of obfuscation techniques and evaluate them under a unified framework. Specifically, we categorize existing obfuscation methods into three major classes (layout, data flow, and control flow) covering 11 subcategories and 19 concrete techniques. We implement these techniques across four programming languages (Solidity, C, C++, and Python) using a consistent LLM-driven approach, and evaluate their effects on 15 LLMs spanning four model families (DeepSeek, OpenAI, Qwen, and LLaMA), as well as on two coding agents (GitHub Copilot and Codex). Our findings reveal both positive and negative impacts of code obfuscation on LLM-based vulnerability detection, highlighting conditions under which obfuscation leads to performance improvements or degradations. We further analyze these outcomes with respect to vulnerability characteristics, code properties, and model attributes. Finally, we outline several open problems and propose future directions to enhance the robustness of LLMs for real-world vulnerability detection.