# On Trojan Signatures in Large Language Models of Code

arXiv: https://arxiv.org/abs/2402.16896 · HTML: https://mlsanity.com/paper/2402.16896/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 22 references that are in the map.
- First posted on arXiv 23 Feb 2024.

## Builds on (most cited first, 12 of 12)

- [Spectral Signatures in Backdoor Attacks](https://mlsanity.com/paper/1811.00636/) (2018)
- [Backdoor Learning: A Survey](https://mlsanity.com/paper/2007.08745/) (2020)
- [Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networks](https://mlsanity.com/paper/1909.03496/) (2019)
- [ONION: A Simple and Effective Defense Against Textual Backdoor Attacks](https://mlsanity.com/paper/2011.10369/) (2020)
- [You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completion](https://mlsanity.com/paper/2007.02220/) (2020)
- [Mitigating backdoor attacks in LSTM-based Text Classification Systems by Backdoor Keyword Identification](https://mlsanity.com/paper/2007.12070/) (2020)
- [Backdoors in Neural Models of Source Code](https://mlsanity.com/paper/2006.06841/) (2020)
- [Backdooring Neural Code Search](https://mlsanity.com/paper/2305.17506/) (2023)
- [Trojan Signatures in DNN Weights](https://mlsanity.com/paper/2109.02836/) (2021)
- [Occlusion-based Detection of Trojan-triggering Inputs in Large Language Models of Code](https://mlsanity.com/paper/2312.04004/) (2023)
- [A Survey of Trojans in Neural Models of Source Code: Taxonomy and Techniques](https://mlsanity.com/paper/2305.03803/) (2023)
- [BadCS: A Backdoor Attack Framework for Code search](https://mlsanity.com/paper/2305.05503/) (2023)

## Built on by (most cited first, 1 of 1)

- [Measuring Impacts of Poisoning on Model Parameters and Embeddings for Large Language Models of Code](https://mlsanity.com/paper/2405.11466/) (2024)

## Abstract (arXiv)

Trojan signatures, as described by Fields et al. (2021), are noticeable differences in the distribution of the trojaned class parameters (weights) and the non-trojaned class parameters of the trojaned model, that can be used to detect the trojaned model. Fields et al. (2021) found trojan signatures in computer vision classification tasks with image models, such as, Resnet, WideResnet, Densenet, and VGG. In this paper, we investigate such signatures in the classifier layer parameters of large language models of source code. Our results suggest that trojan signatures could not generalize to LLMs of code. We found that trojaned code models are stubborn, even when the models were poisoned under more explicit settings (finetuned with pre-trained weights frozen). We analyzed nine trojaned models for two binary classification tasks: clone and defect detection. To the best of our knowledge, this is the first work to examine weight-based trojan signature revelation techniques for large-language models of code and furthermore to demonstrate that detecting trojans only from the weights in such models is a hard problem.