# Eliminating Backdoors in Neural Code Models for Secure Code Understanding arXiv: https://arxiv.org/abs/2408.04683 ยท HTML: https://mlsanity.com/paper/2408.04683/ ## Key facts - Cited by 5 papers in the map (data through 14 Oct 2026). - Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security. - Builds on 28 of its 61 references that are in the map. - First posted on arXiv 8 Aug 2024. ## Placed in - LLM safety and security / [Backdoor and data poisoning attacks](https://mlsanity.com/safety/backdoor-and-data-poisoning-attacks/) ## Builds on (most cited first, 28 of 28) - [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016) - [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023) - [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023) - [CodeBERT: A Pre-Trained Model for Programming and Natural Languages](https://mlsanity.com/paper/2002.08155/) (2020) - [CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation](https://mlsanity.com/paper/2109.00859/) (2021) - [StarCoder: may the source be with you!](https://mlsanity.com/paper/2305.06161/) (2023) - [CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation](https://mlsanity.com/paper/2102.04664/) (2021) - [Universal Adversarial Triggers for Attacking and Analyzing NLP](https://mlsanity.com/paper/1908.07125/) (2019) - [CodeSearchNet Challenge: Evaluating the State of Semantic Code Search](https://mlsanity.com/paper/1909.09436/) (2019) - [Spectral Signatures in Backdoor Attacks](https://mlsanity.com/paper/1811.00636/) (2018) - [Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering](https://mlsanity.com/paper/1811.03728/) (2018) - [UniXcoder: Unified Cross-Modal Pre-training for Code Representation](https://mlsanity.com/paper/2203.03850/) (2022) - [Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networks](https://mlsanity.com/paper/1909.03496/) (2019) - [ONION: A Simple and Effective Defense Against Textual Backdoor Attacks](https://mlsanity.com/paper/2011.10369/) (2020) - [You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completion](https://mlsanity.com/paper/2007.02220/) (2020) - [Adversarial Examples for Models of Code](https://mlsanity.com/paper/1910.07517/) (2019) - [Stealthy Backdoor Attack for Code Models](https://mlsanity.com/paper/2301.02496/) (2023) - [Multi-Modal Attention Network Learning for Semantic Source Code Retrieval](https://mlsanity.com/paper/1909.13516/) (2019) - [Backdoors in Neural Models of Source Code](https://mlsanity.com/paper/2006.06841/) (2020) - [Distilled GPT for Source Code Summarization](https://mlsanity.com/paper/2308.14731/) (2023) - [Defending against Insertion-based Textual Backdoor Attacks via Attribution](https://mlsanity.com/paper/2305.02394/) (2023) - [Enhancing Code Generation Performance of Smaller Models by Distilling the Reasoning Ability of LLMs](https://mlsanity.com/paper/2403.13271/) (2024) - [Code Search based on Context-aware Code Translation](https://mlsanity.com/paper/2202.08029/) (2022) - [Mitigating Backdoor Attack by Injecting Proactive Defensive Backdoor](https://mlsanity.com/paper/2405.16112/) (2024) - [Occlusion-based Detection of Trojan-triggering Inputs in Large Language Models of Code](https://mlsanity.com/paper/2312.04004/) (2023) - [Mutual Information Guided Backdoor Mitigation for Pre-trained Encoders](https://mlsanity.com/paper/2406.03508/) (2024) - [A Survey of Source Code Search: A 3-Dimensional Perspective](https://mlsanity.com/paper/2311.07107/) (2023) - [DistillSeq: A Framework for Safety Alignment Testing in Large Language Models using Knowledge Distillation](https://mlsanity.com/paper/2407.10106/) (2024) ## Built on by (most cited first, 5 of 5) - [Knowledge-Based Multi-Agent Framework for Automated Software Architecture Design](https://mlsanity.com/paper/2503.20536/) (2025) - [Cut the Deadwood Out: Backdoor Purification via Guided Module Substitution](https://mlsanity.com/paper/2412.20476/) (2024) - [Transferable Backdoor Attacks for Code Models via Sharpness-Aware Adversarial Perturbation](https://mlsanity.com/paper/2602.11213/) (2026) - [Securing Code Understanding: Detecting Natural Backdoor Vulnerability in Code Language Models](https://mlsanity.com/paper/2606.10846/) (2026) - [Backdooring Sparse Autoencoders](https://mlsanity.com/paper/2610.06049/) (2026) ## Abstract (arXiv) Neural code models (NCMs) have been widely used to address various code understanding tasks, such as defect detection. However, numerous recent studies reveal that such models are vulnerable to backdoor attacks. Backdoored NCMs function normally on normal/clean code snippets, but exhibit adversary-expected behavior on poisoned code snippets injected with the adversary-crafted trigger. It poses a significant security threat. Therefore, there is an urgent need for effective techniques to detect and eliminate backdoors stealthily implanted in NCMs. To address this issue, in this paper, we innovatively propose a backdoor elimination technique for secure code understanding, called EliBadCode. EliBadCode eliminates backdoors in NCMs by inverting/reverse-engineering and unlearning backdoor triggers. Specifically, EliBadCode first filters the model vocabulary for trigger tokens based on the naming conventions of specific programming languages to reduce the trigger search space and cost. Then, EliBadCode introduces a sample-specific trigger position identification method, which can reduce the interference of non-backdoor (adversarial) perturbations for subsequent trigger inversion, thereby producing effective inverted backdoor triggers efficiently. Backdoor triggers can be viewed as backdoor (adversarial) perturbations. Subsequently, EliBadCode employs a Greedy Coordinate Gradient algorithm to optimize the inverted trigger and designs a trigger anchoring method to purify the inverted trigger. Finally, EliBadCode eliminates backdoors through model unlearning. We evaluate the effectiveness of EliBadCode in eliminating backdoors implanted in multiple NCMs used for three safety-critical code understanding tasks. The results demonstrate that EliBadCode can effectively eliminate backdoors while having minimal adverse effects on the normal functionality of the model.