# Occlusion-based Detection of Trojan-triggering Inputs in Large Language Models of Code

arXiv: https://arxiv.org/abs/2312.04004 · HTML: https://mlsanity.com/paper/2312.04004/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Dec 2023.

## Built on by (most cited first, 9 of 9)

- [An LLM-Assisted Easy-to-Trigger Backdoor Attack on Code Completion Models: Injecting Disguised Vulnerabilities against Strong Detection](https://mlsanity.com/paper/2406.06822/) (2024)
- [Show Me Your Code! Kill Code Poisoning: A Lightweight Method Based on Code Naturalness](https://mlsanity.com/paper/2502.15830/) (2025)
- [Eliminating Backdoors in Neural Code Models for Secure Code Understanding](https://mlsanity.com/paper/2408.04683/) (2024)
- [CodePurify: Defend Backdoor Attacks on Neural Code Models via Entropy-based Purification](https://mlsanity.com/paper/2410.20136/) (2024)
- [Measuring Impacts of Poisoning on Model Parameters and Neuron Activations: A Case Study of Poisoning CodeBERT](https://mlsanity.com/paper/2402.12936/) (2024)
- [On Trojan Signatures in Large Language Models of Code](https://mlsanity.com/paper/2402.16896/) (2024)
- [Backdoors in Code Summarizers: How Bad Is It?](https://mlsanity.com/paper/2506.01825/) (2025)
- [Securing Code Understanding: Detecting Natural Backdoor Vulnerability in Code Language Models](https://mlsanity.com/paper/2606.10846/) (2026)
- [LoRAScan: Detecting Backdoor Prompts in Low-Rank Adapters for Large Language Models via Down-Projection Activation Spikes](https://mlsanity.com/paper/2608.06795/) (2026)

## Abstract (arXiv)

Large language models (LLMs) are becoming an integrated part of software development. These models are trained on large datasets for code, where it is hard to verify each data point. Therefore, a potential attack surface can be to inject poisonous data into the training data to make models vulnerable, aka trojaned. It can pose a significant threat by hiding manipulative behaviors inside models, leading to compromising the integrity of the models in downstream tasks. In this paper, we propose an occlusion-based human-in-the-loop technique, OSeql, to distinguish trojan-triggering inputs of code. The technique is based on the observation that trojaned neural models of code rely heavily on the triggering part of input; hence, its removal would change the confidence of the models in their prediction substantially. Our results suggest that OSeql can detect the triggering inputs with almost 100% recall. We discuss the problem of false positives and how to address them. These results provide a baseline for future studies in this field.