# "Digital Camouflage": The LLVM Challenge in LLM-Based Malware Detection

arXiv: https://arxiv.org/abs/2509.16671 · HTML: https://mlsanity.com/paper/2509.16671/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Sep 2025.

## Built on by (most cited first, 5 of 5)

- [Securing Multi-Agent Systems Against Corruptions via Node Contribution Backpropagation](https://mlsanity.com/paper/2510.19420/) (2025)
- [Small, Free, and Effective: Orchestrating Open-Weight Small Language Models to Outperform Single LLM for Malware Analysis](https://mlsanity.com/paper/2607.20216/) (2026)
- [Poisoned Identifiers Survive LLM Deobfuscation: A Case Study on Claude Opus 4.6](https://mlsanity.com/paper/2604.04289/) (2026)
- [LCC-LLM: Leveraging Code-Centric Large Language Models for Malware Attribution](https://mlsanity.com/paper/2605.05807/) (2026)
- [An Empirical Evaluation of Prompt Injection Vulnerabilities in Large Language Models Across Multilingual and Obfuscated Attack Scenarios](https://mlsanity.com/paper/2606.29602/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have emerged as promising tools for malware detection by analyzing code semantics, identifying vulnerabilities, and adapting to evolving threats. However, their reliability under adversarial compiler-level obfuscation is yet to be discovered. In this study, we empirically evaluate the robustness of three state-of-the-art LLMs: ChatGPT-4o, Gemini Flash 2.5, and Claude Sonnet 4 against compiler-level obfuscation techniques implemented via the LLVM infrastructure. These include control flow flattening, bogus control flow injection, instruction substitution, and split basic blocks, which are widely used to evade detection while preserving malicious behavior. We perform a structured evaluation on 40~C functions (20 vulnerable, 20 secure) sourced from the Devign dataset and obfuscated using LLVM passes. Our results show that these models often fail to correctly classify obfuscated code, with precision, recall, and F1-score dropping significantly after transformation. This reveals a critical limitation: LLMs, despite their language understanding capabilities, can be easily misled by compiler-based obfuscation strategies. To promote reproducibility, we release all evaluation scripts, prompts, and obfuscated code samples in a public repository. We also discuss the implications of these findings for adversarial threat modeling, and outline future directions such as software watermarking, compiler-aware defenses, and obfuscation-resilient model design.