# Preemptive Answer "Attacks" on Chain-of-Thought Reasoning

arXiv: https://arxiv.org/abs/2405.20902 · HTML: https://mlsanity.com/paper/2405.20902/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning.
- Builds on 31 of its 57 references that are in the map.
- First posted on arXiv 31 May 2024.

## Placed in

- LLM reasoning / [Prompting and chain-of-thought reasoning](https://mlsanity.com/lr/prompting-and-chain-of-thought-reasoning/)

## Builds on (most cited first, 30 of 31)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://mlsanity.com/paper/2202.12837/) (2022)
- [HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal](https://mlsanity.com/paper/2402.04249/) (2024)
- [Jailbreaking Black Box Large Language Models in Twenty Queries](https://mlsanity.com/paper/2310.08419/) (2023)
- [Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks](https://mlsanity.com/paper/2211.12588/) (2022)
- [Large Language Models Cannot Self-Correct Reasoning Yet](https://mlsanity.com/paper/2310.01798/) (2023)
- [Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions](https://mlsanity.com/paper/2212.10509/) (2022)
- [Graph of Thoughts: Solving Elaborate Problems with Large Language Models](https://mlsanity.com/paper/2308.09687/) (2023)
- [AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models](https://mlsanity.com/paper/2310.04451/) (2023)
- [DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models](https://mlsanity.com/paper/2306.11698/) (2023)
- [Jailbreaking ChatGPT via Prompt Engineering: An Empirical Study](https://mlsanity.com/paper/2305.13860/) (2023)
- [Is ChatGPT a General-Purpose Natural Language Processing Task Solver?](https://mlsanity.com/paper/2302.06476/) (2023)
- [Large Language Models Are State-of-the-Art Evaluators of Translation Quality](https://mlsanity.com/paper/2302.14520/) (2023)
- [Certifying LLM Safety against Adversarial Prompting](https://mlsanity.com/paper/2309.02705/) (2023)
- [PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Prompts](https://mlsanity.com/paper/2306.04528/) (2023)
- [LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examples](https://mlsanity.com/paper/2310.01469/) (2023)
- [SelfCheck: Using LLMs to Zero-Shot Check Their Own Step-by-Step Reasoning](https://mlsanity.com/paper/2308.00436/) (2023)
- [The Earth is Flat because...: Investigating LLMs' Belief towards Misinformation via Persuasive Conversation](https://mlsanity.com/paper/2312.09085/) (2023)
- [Prompt as Triggers for Backdoor Attack: Examining the Vulnerability in Language Models](https://mlsanity.com/paper/2305.01219/) (2023)
- [Context-faithful Prompting for Large Language Models](https://mlsanity.com/paper/2303.11315/) (2023)
- [Complementary Explanations for Effective In-Context Learning](https://mlsanity.com/paper/2211.13892/) (2022)
- [Adversarial Demonstration Attacks on Large Language Models](https://mlsanity.com/paper/2305.14950/) (2023)
- [Reprompting: Automated Chain-of-Thought Prompt Inference Through Gibbs Sampling](https://mlsanity.com/paper/2305.09993/) (2023)
- [Summarization is (Almost) Dead](https://mlsanity.com/paper/2309.09558/) (2023)
- [NOTABLE: Transferable Backdoor Attacks Against Prompt-based NLP Models](https://mlsanity.com/paper/2305.17826/) (2023)

## Built on by (most cited first, 19 of 19)

- [Hierarchical Reasoning Model](https://mlsanity.com/paper/2506.21734/) (2025)
- [The Landscape of Prompt Injection Threats in LLM Agents: From Taxonomy to Analysis](https://mlsanity.com/paper/2602.10453/) (2026)
- [Investigating Mysteries of CoT-Augmented Distillation](https://mlsanity.com/paper/2406.14511/) (2024)
- [Robust Answers, Fragile Logic: Probing the Decoupling Hypothesis in LLM Reasoning](https://mlsanity.com/paper/2505.17406/) (2025)
- [Stepwise Reasoning Error Disruption Attack of LLMs](https://mlsanity.com/paper/2412.11934/) (2024)
- [Don't Take the Premise for Granted: Evaluating the Premise Critique Ability of Large Language Models](https://mlsanity.com/paper/2505.23715/) (2025)
- [Boosting LLM Translation Skills without General Ability Loss via Rationale Distillation](https://mlsanity.com/paper/2410.13944/) (2024)
- [Make LLMs better zero-shot reasoners: Structure-orientated autonomous reasoning](https://mlsanity.com/paper/2410.19000/) (2024)
- [SoK: Systematizing LLM Prompt Security: Taxonomies, Datasets, and Unified Evaluation of Attacks and Defenses](https://mlsanity.com/paper/2510.15476/) (2025)
- [TRAP: Hijacking VLA CoT-Reasoning via Adversarial Patches](https://mlsanity.com/paper/2603.23117/) (2026)
- [Exploring the Impact of Instruction-Tuning on LLM's Susceptibility to Misinformation](https://mlsanity.com/paper/2507.18203/) (2025)
- [CoTDeceptor:Adversarial Code Obfuscation Against CoT-Enhanced LLM Code Agents](https://mlsanity.com/paper/2512.21250/) (2025)
- [Less Diverse, Less Safe: The Indirect But Pervasive Risk of Test-Time Scaling in Large Language Models](https://mlsanity.com/paper/2510.08592/) (2025)
- [TraceGuard: Process-Guided Firewall against Reasoning Backdoors in Large Language Models](https://mlsanity.com/paper/2603.02436/) (2026)
- [Beyond Content Safety: Real-Time Monitoring for Reasoning Vulnerabilities in Large Language Models](https://mlsanity.com/paper/2603.25412/) (2026)
- [Unreal Thinking: Chain-of-Thought Hijacking via Two-stage Backdoor](https://mlsanity.com/paper/2604.09235/) (2026)
- [DREAM: Dynamic Red-teaming across Environments for AI Models](https://mlsanity.com/paper/2512.19016/) (2025)
- [Altered Thoughts, Altered Actions: Probing Chain-of-Thought Vulnerabilities in VLA Robotic Manipulation](https://mlsanity.com/paper/2603.12717/) (2026)
- [The Dual-use Dilemma in LLMs: Do Empowering Ethical Capacities Make a Degraded Utility?](https://mlsanity.com/paper/2501.13952/) (2025)

## Abstract (arXiv)

Large language models (LLMs) showcase impressive reasoning capabilities when coupled with Chain-of-Thought (CoT) prompting. However, the robustness of this approach warrants further investigation. In this paper, we introduce a novel scenario termed preemptive answers, where the LLM obtains an answer before engaging in reasoning. This situation can arise inadvertently or induced by malicious users by prompt injection attacks. Experiments reveal that preemptive answers significantly impair the model's reasoning capability across various CoT methods and a broad spectrum of datasets. To bolster the robustness of reasoning, we propose two measures aimed at mitigating this issue to some extent.