mlsanityData to 7 Oct 2026

Preemptive Answer "Attacks" on Chain-of-Thought Reasoning

arXiv 2405.20902 ↗First posted 31 May 2024cs.CL cs.AI cs.CRcited by 16

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Large language models (LLMs) showcase impressive reasoning capabilities when coupled with Chain-of-Thought (CoT) prompting. However, the robustness of this approach warrants further investigation. In this paper, we introduce a novel scenario termed preemptive answers, where the LLM obtains an answer before engaging in reasoning. This situation can arise inadvertently or induced by malicious users by prompt injection attacks. Experiments reveal that preemptive answers significantly impair the model's reasoning capability across various CoT methods and a broad spectrum of datasets. To bolster the robustness of reasoning, we propose two measures aimed at mitigating this issue to some extent.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 31 of its 57 references that are in the map. First posted on arXiv 31 May 2024.

Builds on

31 of 57 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023Stress Testing Chain-of-Thought Prompting for Large Language Models28 Sep
  2. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Models18 Aug
  3. 2023Reprompting: Automated Chain-of-Thought Prompt Inference Through Gibbs Sampling17 May
  4. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks22 Nov

Other references · 27

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  5. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  6. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  7. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  8. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  9. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  10. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 965
  11. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 911
  12. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 738
15 more
  1. 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 505
  2. 2023Is ChatGPT a General-Purpose Natural Language Processing Task Solver?cited by 486
  3. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 478
  4. 2023Large Language Models Are State-of-the-Art Evaluators of Translation Qualitycited by 416
  5. 2023Certifying LLM Safety against Adversarial Promptingcited by 248
  6. 2023PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Promptscited by 218
  7. 2023LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examplescited by 177
  8. 2023SelfCheck: Using LLMs to Zero-Shot Check Their Own Step-by-Step Reasoningcited by 157
  9. 2023The Earth is Flat because...: Investigating LLMs' Belief towards Misinformation via Persuasive Conversationcited by 107
  10. 2023Prompt as Triggers for Backdoor Attack: Examining the Vulnerability in Language Modelscited by 87
  11. 2023Context-faithful Prompting for Large Language Modelscited by 76
  12. 2022Complementary Explanations for Effective In-Context Learningcited by 71
  13. 2023Adversarial Demonstration Attacks on Large Language Modelscited by 58
  14. 2023Summarization is (Almost) Deadcited by 43
  15. 2023NOTABLE: Transferable Backdoor Attacks Against Prompt-based NLP Modelscited by 33

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 8; 2026: 4.

16 papers.

Sort:

By area:

  1. 2025Hierarchical Reasoning Modelcited by 167
  2. 2026The Landscape of Prompt Injection Threats in LLM Agents: From Taxonomy to Analysiscited by 25
  3. 2024Investigating Mysteries of CoT-Augmented Distillationcited by 15
  4. 2025Robust Answers, Fragile Logic: Probing the Decoupling Hypothesis in LLM Reasoningcited by 11
  5. 2024Stepwise Reasoning Error Disruption Attack of LLMscited by 8
  6. 2025Don't Take the Premise for Granted: Evaluating the Premise Critique Ability of Large Language Modelscited by 8
  7. 2024Make LLMs better zero-shot reasoners: Structure-orientated autonomous reasoningcited by 6
  8. 2024Boosting LLM Translation Skills without General Ability Loss via Rationale Distillationcited by 5
  9. 2026TRAP: Hijacking VLA CoT-Reasoning via Adversarial Patchescited by 5
  10. 2025Exploring the Impact of Instruction-Tuning on LLM's Susceptibility to Misinformationcited by 4
  11. 2025SoK: Systematizing LLM Prompt Security: Taxonomies, Datasets, and Unified Evaluation of Attacks and Defensescited by 4
  12. 2025Less Diverse, Less Safe: The Indirect But Pervasive Risk of Test-Time Scaling in Large Language Modelscited by 2
4 more
  1. 2025CoTDeceptor:Adversarial Code Obfuscation Against CoT-Enhanced LLM Code Agentscited by 2
  2. 2026Beyond Content Safety: Real-Time Monitoring for Reasoning Vulnerabilities in Large Language Modelscited by 2
  3. 2025The Dual-use Dilemma in LLMs: Do Empowering Ethical Capacities Make a Degraded Utility?cited by 0
  4. 2026Altered Thoughts, Altered Actions: Probing Chain-of-Thought Vulnerabilities in VLA Robotic Manipulationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Chain-of-Defensive-Thought: Structured Reasoning Elicits Robustness in Large Language Models against Reference Corruption
  2. 2024Chain-of-Thought Reasoning Without Prompting
  3. 2026Fragile Thoughts: How Large Language Models Handle Chain-of-Thought Perturbations
  4. 2024ChainLM: Empowering Large Language Models with Improved Chain-of-Thought Prompting
  5. 2026Thought-Transfer: Indirect Targeted Poisoning Attacks on Chain-of-Thought Reasoning Models
  6. 2023It's Not Easy Being Wrong: Large Language Models Struggle with Process of Elimination Reasoning
  7. 2025Unveiling Confirmation Bias in Chain-of-Thought Reasoning
  8. 2022Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters