DarkMind: Latent Chain-of-Thought Backdoor in Customized LLMs
On the map: LLM reasoning → Reasoning for agents, planning and code
also in 1 more
Abstract (arXiv)
With the rapid rise of personalized AI, customized large language models (LLMs) equipped with Chain of Thought (COT) reasoning now power millions of AI agents. However, their complex reasoning processes introduce new and largely unexplored security vulnerabilities. We present DarkMind, a novel latent reasoning level backdoor attack that targets customized LLMs by manipulating internal COT steps without altering user queries. Unlike prior prompt based attacks, DarkMind activates covertly within the reasoning chain via latent triggers, enabling adversarial behaviors without modifying input prompts or requiring access to model parameters. To achieve stealth and reliability, we propose dual trigger types instant and retrospective and integrate them within a unified embedding template that governs trigger dependent activation, employ a stealth optimization algorithm to minimize semantic drift, and introduce an automated conversation starter for covert activation across domains. Comprehensive experiments on eight reasoning datasets spanning arithmetic, commonsense, and symbolic domains, using five LLMs, demonstrate that DarkMind consistently achieves high attack success rates. We further investigate defense strategies to mitigate these risks and reveal that reasoning level backdoors represent a significant yet underexplored threat, underscoring the need for robust, reasoning aware security mechanisms.
Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 37 of its 50 references that are in the map. First posted on arXiv 24 Jan 2025.
Builds on
37 of 50 references.
Closest ancestors · 1
Same direction, newest first
Other references · 36
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2021Training Verifiers to Solve Math Word Problemscited by 9,743
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
- 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
- 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
- 2021Program Synthesis with Large Language Modelscited by 3,509
- 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
- 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
- 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
24 more
- 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 911
- 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
- 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
- 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
- 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 505
- 2021A Diverse Corpus for Evaluating and Developing English Math Word Problem Solverscited by 314
- 2024Large Language Models for Mathematical Reasoning: Progresses and Challengescited by 283
- 2022Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matterscited by 221
- 2024Advancing LLM Reasoning Generalists with Preference Treescited by 191
- 2020Blind Backdoors in Deep Learning Modelscited by 174
- 2023GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Modelscited by 174
- 2020Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defensescited by 171
- 2022Language Models of Code are Few-Shot Commonsense Learnerscited by 160
- 2023MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoningcited by 155
- 2020Dynamic Backdoor Attacks Against Machine Learning Modelscited by 147
- 2020RAB: Provable Robustness Against Backdoor Attackscited by 115
- 2022Narcissus: A Practical Clean-Label Backdoor Attack with Limited Informationcited by 102
- 2023Backdoor Attacks for In-Context Learning with Language Modelscited by 92
- 2024We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMscited by 61
- 2022BadPrompt: Backdoor Attacks on Continuous Promptscited by 59
- 2024Instruction Backdoor Attacks Against Customized LLMscited by 50
- 2022SemAttack: Natural Textual Attacks via Different Semantic Spacescited by 42
- 2024Scalable Qualitative Coding with LLMs: Chain-of-Thought Reasoning Matches Human Performance in Some Hermeneutic Taskscited by 22
- 2024Persistent Backdoor Attacks in Continual Learningcited by 11
Built on by
Citing papers per year: 2025: 11; 2026: 4.
15 papers.
Sort:
By area:
- 2025Large Language Model Agent: A Survey on Methodology, Applications and Challengescited by 194
- 2025ShadowCoT: Cognitive Hijacking for Stealthy Reasoning Backdoors in LLMscited by 23
- 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Modelscited by 22
- 2025Backdoor Attribution: Elucidating and Controlling Backdoor in Language Modelscited by 8
- 2025BadThink: Triggered Overthinking Attacks on Chain-of-Thought Reasoning in Large Language Modelscited by 7
- 2025Thought Purity: A Defense Framework For Chain-of-Thought Attackcited by 3
- 2025Reasoning Introduces New Poisoning Attacks Yet Makes Them More Complicatedcited by 3
- 2025Beyond SFT: Reinforcement Learning for Safer Large Reasoning Models with Better Reasoning Abilitycited by 2
- 2026Thought-Transfer: Indirect Targeted Poisoning Attacks on Chain-of-Thought Reasoning Modelscited by 2
- 2025Rethinking Reasoning: A Survey on Reasoning-based Backdoors in LLMscited by 1
- 2025From Perception to Reasoning: Deep Thinking Empowers Multimodal Large Language Modelscited by 1
- 2026Thinking Wrong in Silence: Backdoor Attacks on Continuous Latent Reasoningcited by 1
- 2026OPBackdoor: Opportunistic Backdoors via Alibi-Aligned Reasoning21 Sep 2026
- 2026Alignment of LRMs via Counter-Aligned Few-Shot Conversation Exposure16 Aug 2026
- 2026Thinking Wrong in Silence: Backdoor Attacks on Continuous Latent Reasoning1 Apr 2026
- 2026Thought-Transfer: Indirect Targeted Poisoning Attacks on Chain-of-Thought Reasoning Models27 Jan 2026
- 2025Beyond SFT: Reinforcement Learning for Safer Large Reasoning Models with Better Reasoning Ability1 Dec 2025
- 2025From Perception to Reasoning: Deep Thinking Empowers Multimodal Large Language Models17 Nov 2025
- 2025BadThink: Triggered Overthinking Attacks on Chain-of-Thought Reasoning in Large Language Models13 Nov 2025
- 2025Rethinking Reasoning: A Survey on Reasoning-based Backdoors in LLMs9 Oct 2025
- 2025Backdoor Attribution: Elucidating and Controlling Backdoor in Language Models26 Sep 2025
- 2025Reasoning Introduces New Poisoning Attacks Yet Makes Them More Complicated6 Sep 2025
- 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Models4 Sep 2025
- 2025A Real-Time, Self-Tuning Moderator Framework for Adversarial Prompt Detection10 Aug 2025
- 2025ShadowCoT: Cognitive Hijacking for Stealthy Reasoning Backdoors in LLMscited by 23
- 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Modelscited by 22
- 2025BadThink: Triggered Overthinking Attacks on Chain-of-Thought Reasoning in Large Language Modelscited by 7
- 2025Thought Purity: A Defense Framework For Chain-of-Thought Attackcited by 3
- 2025Reasoning Introduces New Poisoning Attacks Yet Makes Them More Complicatedcited by 3
- 2025Beyond SFT: Reinforcement Learning for Safer Large Reasoning Models with Better Reasoning Abilitycited by 2
- 2026Thought-Transfer: Indirect Targeted Poisoning Attacks on Chain-of-Thought Reasoning Modelscited by 2
- 2025Rethinking Reasoning: A Survey on Reasoning-based Backdoors in LLMscited by 1
- 2025From Perception to Reasoning: Deep Thinking Empowers Multimodal Large Language Modelscited by 1
- 2026Thinking Wrong in Silence: Backdoor Attacks on Continuous Latent Reasoningcited by 1
- 2026Alignment of LRMs via Counter-Aligned Few-Shot Conversation Exposurecited by 0
- 2026Alignment of LRMs via Counter-Aligned Few-Shot Conversation Exposure16 Aug 2026
- 2026Thinking Wrong in Silence: Backdoor Attacks on Continuous Latent Reasoning1 Apr 2026
- 2026Thought-Transfer: Indirect Targeted Poisoning Attacks on Chain-of-Thought Reasoning Models27 Jan 2026
- 2025Beyond SFT: Reinforcement Learning for Safer Large Reasoning Models with Better Reasoning Ability1 Dec 2025
- 2025From Perception to Reasoning: Deep Thinking Empowers Multimodal Large Language Models17 Nov 2025
- 2025BadThink: Triggered Overthinking Attacks on Chain-of-Thought Reasoning in Large Language Models13 Nov 2025
- 2025Rethinking Reasoning: A Survey on Reasoning-based Backdoors in LLMs9 Oct 2025
- 2025Reasoning Introduces New Poisoning Attacks Yet Makes Them More Complicated6 Sep 2025
- 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Models4 Sep 2025
- 2025Thought Purity: A Defense Framework For Chain-of-Thought Attack16 Jul 2025
- 2025ShadowCoT: Cognitive Hijacking for Stealthy Reasoning Backdoors in LLMs8 Apr 2025
Similar, not cited
Close in wording, no citation either way
- 2026Critical-CoT: A Robust Defense Framework against Reasoning-Level Backdoor Attacks in Large Language Models
- 2024Chain-of-Scrutiny: Detecting Backdoor Attacks for Large Language Models
- 2025AutoBackdoor: Automating Backdoor Attacks via LLM Agents
- 2026STAR: Detecting Inference-time Backdoors in LLM Reasoning via State-Transition Amplification Ratio
- 2025The dark deep side of DeepSeek: Fine-tuning attacks against the safety alignment of CoT-enabled models
- 2025Thought Crime: Backdoors and Emergent Misalignment in Reasoning Models
- 2025BadReasoner: Planting Tunable Overthinking Backdoors into Large Reasoning Models for Fun or Profit
- 2025One Token Embedding Is Enough to Deadlock Your Large Reasoning Model