mlsanityData to 7 Oct 2026

DarkMind: Latent Chain-of-Thought Backdoor in Customized LLMs

arXiv 2501.18617 ↗First posted 24 Jan 2025cs.CR cs.LGcited by 15

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

With the rapid rise of personalized AI, customized large language models (LLMs) equipped with Chain of Thought (COT) reasoning now power millions of AI agents. However, their complex reasoning processes introduce new and largely unexplored security vulnerabilities. We present DarkMind, a novel latent reasoning level backdoor attack that targets customized LLMs by manipulating internal COT steps without altering user queries. Unlike prior prompt based attacks, DarkMind activates covertly within the reasoning chain via latent triggers, enabling adversarial behaviors without modifying input prompts or requiring access to model parameters. To achieve stealth and reliability, we propose dual trigger types instant and retrospective and integrate them within a unified embedding template that governs trigger dependent activation, employ a stealth optimization algorithm to minimize semantic drift, and introduce an automated conversation starter for covert activation across domains. Comprehensive experiments on eight reasoning datasets spanning arithmetic, commonsense, and symbolic domains, using five LLMs, demonstrate that DarkMind consistently achieves high attack success rates. We further investigate defense strategies to mitigate these risks and reveal that reasoning level backdoors represent a significant yet underexplored threat, underscoring the need for robust, reasoning aware security mechanisms.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 37 of its 50 references that are in the map. First posted on arXiv 24 Jan 2025.

Builds on

37 of 50 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024BadChain: Backdoor Chain-of-Thought Prompting for Large Language Models20 Jan

Other references · 36

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  7. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  8. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  9. 2021Program Synthesis with Large Language Modelscited by 3,509
  10. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  11. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  12. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
24 more
  1. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 911
  2. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
  3. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  4. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
  5. 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 505
  6. 2021A Diverse Corpus for Evaluating and Developing English Math Word Problem Solverscited by 314
  7. 2024Large Language Models for Mathematical Reasoning: Progresses and Challengescited by 283
  8. 2022Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matterscited by 221
  9. 2024Advancing LLM Reasoning Generalists with Preference Treescited by 191
  10. 2020Blind Backdoors in Deep Learning Modelscited by 174
  11. 2023GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Modelscited by 174
  12. 2020Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defensescited by 171
  13. 2022Language Models of Code are Few-Shot Commonsense Learnerscited by 160
  14. 2023MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoningcited by 155
  15. 2020Dynamic Backdoor Attacks Against Machine Learning Modelscited by 147
  16. 2020RAB: Provable Robustness Against Backdoor Attackscited by 115
  17. 2022Narcissus: A Practical Clean-Label Backdoor Attack with Limited Informationcited by 102
  18. 2023Backdoor Attacks for In-Context Learning with Language Modelscited by 92
  19. 2024We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMscited by 61
  20. 2022BadPrompt: Backdoor Attacks on Continuous Promptscited by 59
  21. 2024Instruction Backdoor Attacks Against Customized LLMscited by 50
  22. 2022SemAttack: Natural Textual Attacks via Different Semantic Spacescited by 42
  23. 2024Scalable Qualitative Coding with LLMs: Chain-of-Thought Reasoning Matches Human Performance in Some Hermeneutic Taskscited by 22
  24. 2024Persistent Backdoor Attacks in Continual Learningcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 11; 2026: 4.

15 papers.

Sort:

By area:

Similar, not cited

Close in wording, no citation either way

  1. 2026Critical-CoT: A Robust Defense Framework against Reasoning-Level Backdoor Attacks in Large Language Models
  2. 2024Chain-of-Scrutiny: Detecting Backdoor Attacks for Large Language Models
  3. 2025AutoBackdoor: Automating Backdoor Attacks via LLM Agents
  4. 2026STAR: Detecting Inference-time Backdoors in LLM Reasoning via State-Transition Amplification Ratio
  5. 2025The dark deep side of DeepSeek: Fine-tuning attacks against the safety alignment of CoT-enabled models
  6. 2025Thought Crime: Backdoors and Emergent Misalignment in Reasoning Models
  7. 2025BadReasoner: Planting Tunable Overthinking Backdoors into Large Reasoning Models for Fun or Profit
  8. 2025One Token Embedding Is Enough to Deadlock Your Large Reasoning Model