mlsanityData to 14 Oct 2026

Localizing Malicious Outputs from CodeLLM

arXiv 2509.17070 ↗First posted 21 Sep 2025cs.CR cs.CL cs.LGcited by 0

On the map: LLM safety and security → Backdoor and data poisoning attacks

Abstract (arXiv)

We introduce FreqRank, a mutation-based defense to localize malicious components in LLM outputs and their corresponding backdoor triggers. FreqRank assumes that the malicious sub-string(s) consistently appear in outputs for triggered inputs and uses a frequency-based ranking system to identify them. Our ranking system then leverages this knowledge to localize the backdoor triggers present in the inputs. We create nine malicious models through fine-tuning or custom instructions for three downstream tasks, namely, code completion (CC), code generation (CG), and code summarization (CS), and show that they have an average attack success rate (ASR) of 86.6%. Furthermore, FreqRank's ranking system highlights the malicious outputs as one of the top five suggestions in 98% of cases. We also demonstrate that FreqRank's effectiveness scales as the number of mutants increases and show that FreqRank is capable of localizing the backdoor trigger effectively even with a limited number of triggered samples. Finally, we show that our approach is 35-50% more effective than other defense methods.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security. Builds on 19 of its 28 references that are in the map. First posted on arXiv 21 Sep 2025.

Builds on

19 of 28 references.

Closest ancestors · 7

Same direction, newest first

  1. 2024An LLM-Assisted Easy-to-Trigger Backdoor Attack on Code Completion Models: Injecting Disguised Vulnerabilities against Strong Detection10 Jun
  2. 2024Instruction Backdoor Attacks Against Customized LLMs14 Feb
  3. 2023Backdoor Attacks for In-Context Learning with Language Models27 Jul
  4. 2023ChatGPT as an Attack Tool: Stealthy Textual Backdoor Attack via Blackbox Generative Model Trigger27 Apr
  5. 2023BadGPT: Exploring Security Vulnerabilities of ChatGPT via Backdoor Attacks to InstructGPT21 Feb
  6. 2023Stealthy Backdoor Attack for Code Models6 Jan
  7. 2021RAP: Robustness-Aware Perturbations for Defending against Backdoor Attacks on NLP Models15 Oct

Other references · 12

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  2. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  3. 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 654
  4. 2019STRIP: A Defence Against Trojan Attacks on Deep Neural Networkscited by 459
  5. 2020ONION: A Simple and Effective Defense Against Textual Backdoor Attackscited by 263
  6. 2024Chain of Thought Empowers Transformers to Solve Inherently Serial Problemscited by 260
  7. 2020BadNL: Backdoor Attacks against NLP Models with Semantic-preserving Improvementscited by 211
  8. 2024CodeGemma: Open Code Models Based on Gemmacited by 150
  9. 2019Design and Evaluation of a Multi-Domain Trojan Detection Method on Deep Neural Networkscited by 71
  10. 2019Model Agnostic Defence against Backdoor Attacks in Machine Learningcited by 52
  11. 2020Backdoors in Neural Models of Source Codecited by 41
  12. 2024Shortcuts Everywhere and Nowhere: Exploring Multi-Trigger Backdoor Attackscited by 19

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025ImportSnare: Directed "Code Manual" Hijacking in Retrieval-Augmented Code Generation
  2. 2026SecureForge: Finding and Preventing Vulnerabilities in LLM-Generated Code via Prompt Optimization
  3. 2026Beware What You Autocomplete: Forensic Attribution of Backdoored Code Completions
  4. 2023TrojanedCM: A Repository of Trojaned Large Language Models of Code
  5. 2025MGC: A Compiler Framework Exploiting Compositional Blindness in Aligned LLMs for Malware Generation
  6. 2024CodePurify: Defend Backdoor Attacks on Neural Code Models via Entropy-based Purification
  7. 2024Double Backdoored: Converting Code Large Language Model Backdoors to Traditional Malware via Adversarial Instruction Tuning Attacks
  8. 2025"Digital Camouflage": The LLVM Challenge in LLM-Based Malware Detection