A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Models
On the map: LLM reasoning → Reasoning for agents, planning and code
Abstract (arXiv)
The development of Long-CoT reasoning has advanced LLM performance across various tasks, including language understanding, complex problem solving, and code generation. This paradigm enables models to generate intermediate reasoning steps, thereby improving both accuracy and interpretability. However, despite these advancements, a comprehensive understanding of how CoT-based reasoning affects the trustworthiness of language models remains underdeveloped. In this paper, we survey recent work on reasoning models and CoT techniques, focusing on five core dimensions of trustworthy reasoning: truthfulness, safety, robustness, fairness, and privacy. For each aspect, we provide a clear and structured overview of recent studies in chronological order, along with detailed analyses of their methodologies, findings, and limitations. Future research directions are also appended at the end for reference and discussion. Overall, while reasoning techniques hold promise for enhancing model trustworthiness through hallucination mitigation, harmful content detection, and robustness improvement, cutting-edge reasoning models themselves often suffer from comparable or even greater vulnerabilities in safety, robustness, and privacy. By synthesizing these insights, we hope this work serves as a valuable and timely resource for the AI safety community to stay informed on the latest progress in reasoning trustworthiness. A full list of related papers can be found at hrefhttps://github.com/ybwang119/Awesome-reasoning-safetyhttps://github.com/ybwang119/Awesome-reasoning-safety.
Key facts. Cited by 25 papers in the map (data through 14 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 232 of its 365 references that are in the map. First posted on arXiv 4 Sep 2025.
Builds on
232 of 365 references.
Closest ancestors · 66
Same direction, newest first
- 2025Reasoning as an Adaptive Defense for Safety1 Jul
- 2025Leaky Thoughts: Large Reasoning Models Are Not Private Thinkers18 Jun
- 2025Excessive Reasoning Attack on Reasoning LLMs17 Jun
- 2025Weakest Link in the Chain: Security Vulnerabilities in Advanced Reasoning Models16 Jun
- 2025Thought Crime: Backdoors and Emergent Misalignment in Reasoning Models16 Jun
- 2025Reasoning Model Unlearning: Forgetting Traces, Not Just Answers, While Preserving Reasoning Skills15 Jun
- 2025Step-by-Step Reasoning Attack: Revealing 'Erased' Knowledge in Large Language Models14 Jun
- 2025Benchmarking Gaslighting Negation Attacks Against Reasoning Models11 Jun
- 2025RSafe: Incentivizing proactive reasoning to build robust and adaptive LLM safeguards9 Jun
- 2025HauntAttack: When Attack Follows Reasoning as a Shadow8 Jun
- 2025GUARD:Dual-Agent based Backdoor Defense on Chain-of-Thought in Neural Code Generation27 May
- 2025Safety Through Reasoning: An Empirical Study of Reasoning Guardrail Models26 May
38 more
- 2025Beyond Safe Answers: A Benchmark for Evaluating True Risk Awareness in Large Reasoning Models26 May
- 2025VisCRA: A Visual Chain Reasoning Attack for Jailbreaking Multimodal Large Language Models26 May
- 2025ReasoningShield: Safety Detection over Reasoning Traces of Large Reasoning Models22 May
- 2025Three Minds, One Legend: Jailbreak Large Reasoning Model with Adaptive Stacked Ciphers22 May
- 2025SafeKey: Amplifying Aha-Moment Insights for Safety Reasoning22 May
- 2025RRTL: Red Teaming Reasoning Large Language Models in Tool Learning21 May
- 2025How Should We Enhance the Safety of Large Reasoning Models: An Empirical Study21 May
- 2025R-TOFU: Unlearning in Large Reasoning Models21 May
- 2025SAFEPATH: Preventing Harmful Reasoning in Chain-of-Thought via Early Alignment20 May
- 2025Context Reasoner: Incentivizing Reasoning Capability for Contextualized Privacy and Safety Compliance via Reinforcement Learning20 May
- 2025Think Twice Before You Act: Enhancing Agent Behavioral Safety with Thought Correction16 May
- 2025GuardReasoner-VL: Safeguarding VLMs via Reinforced Reasoning16 May
- 2025AutoRAN: Automated Hijacking of Safety Reasoning in Large Reasoning Models16 May
- 2025Practical Reasoning Interruption Attacks on Reasoning Large Language Models10 May
- 2025Think in Safety: Unveiling and Mitigating Safety Alignment Collapse in Multimodal Large Reasoning Model10 May
- 2025Adaptive Social Learning via Mode Policy Optimization for Language Agents4 May
- 2025Chain-of-Defensive-Thought: Structured Reasoning Elicits Robustness in Large Language Models against Reference Corruption29 Apr
- 2025Token-Efficient Prompt Injection Attack: Provoking Cessation in LLM Reasoning via Adaptive Token Compression29 Apr
- 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment22 Apr
- 2025MrGuard: A Multilingual Reasoning Guardrail for Universal LLM Safety21 Apr
- 2025CodeCrash: Exposing LLM Fragility to Misleading Natural Language in Code Reasoning19 Apr
- 2025RealSafe-R1: Safety-Aligned DeepSeek-R1 without Compromising Reasoning Capability14 Apr
- 2025SaRO: Enhancing LLM Safety through Reasoning-based Alignment13 Apr
- 2025SafeMLRM: Demystifying Safety in Multi-modal Large Reasoning Models9 Apr
- 2025ShadowCoT: Cognitive Hijacking for Stealthy Reasoning Backdoors in LLMs8 Apr
- 2025SAFER: Advancing Safety Alignment via Efficient Ex-Ante Reasoning3 Apr
- 2025STAR-1: Safer Alignment of Reasoning LLMs with 1K Data2 Apr
- 2025ShieldAgent: Shielding Agents via Verifiable Safety Policy Reasoning26 Mar
- 2025Process or Result? Manipulated Ending Tokens Can Mislead Reasoning LLMs to Ignore the Correct Reasoning Steps25 Mar
- 2025Safety Evaluation and Enhancement of DeepSeek Models in Chinese Contexts18 Mar
- 2025Safety is Not Only About Refusal: Reasoning-Enhanced Fine-tuning for Interpretable LLM Safety6 Mar
- 2025Cats Confuse Reasoning LLM: Query Agnostic Adversarial Triggers for Reasoning Models3 Mar
- 2025Output Length Effect on DeepSeek-R1's Safety in Forced Thinking2 Mar
- 2025Safety Tax: Safety Alignment Makes Your Large Reasoning Models Less Reasonable1 Mar
- 2025A Mousetrap: Fooling Large Reasoning Models for Jailbreak with Chain of Iterative Chaos19 Feb
- 2025Reasoning-to-Defend: Safety-Aware Reasoning Can Defend Large Language Models from Jailbreaking18 Feb
- 2025H-CoT: Hijacking the Chain-of-Thought Safety Reasoning Mechanism to Jailbreak Large Reasoning Models, Including OpenAI o1/o3, DeepSeek-R1, and Gemini 2.0 Flash Thinking18 Feb
- 2025The Hidden Risks of Large Reasoning Models: A Safety Assessment of R118 Feb
50 newest of 66 All 66 (JSON)
Other references · 166
- 2023GPT-4 Technical Reportcited by 18,396
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2014Explaining and Harnessing Adversarial Examplescited by 7,364
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
- 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
- 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
- 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
- 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
- 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,554
38 more
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
- 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
- 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
- 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 1,053
- 2022Solving math word problems with process- and outcome-based feedbackcited by 901
- 2024DeepSeek-V3 Technical Reportcited by 811
- 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 703
- 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 672
- 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMscited by 570
- 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 529
- 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Modelscited by 464
- 2024LLaVA-CoT: Let Vision Language Models Reason Step-by-Stepcited by 454
- 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 431
- 2025Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approachcited by 415
- 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 384
- 2025Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscationcited by 384
- 2023Orca: Progressive Learning from Complex Explanation Traces of GPT-4cited by 379
- 2024Measuring short-form factuality in large language modelscited by 373
- 2025R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalizationcited by 370
- 2023A Survey of Hallucination in Large Foundation Modelscited by 368
- 2025Chain of Draft: Thinking Faster by Writing Lesscited by 297
- 2025Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Modelscited by 293
- 2025TokenSkip: Controllable Chain-of-Thought Compression in LLMscited by 249
- 2025LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RLcited by 248
- 2024SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decodingcited by 247
- 2025SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Modelscited by 212
- 2025Multimodal Chain-of-Thought Reasoning: A Comprehensive Surveycited by 182
- 2024Mulberry: Empowering MLLM with o1-like Reasoning and Reflection via Collective Monte Carlo Tree Searchcited by 175
- 2025Chain-of-Thought Reasoning In The Wild Is Not Always Faithfulcited by 170
- 2025CoT-Valve: Length-Compressible Chain-of-Thought Tuningcited by 169
- 2024Eight Methods to Evaluate Robust Unlearning in LLMscited by 163
- 2025Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMscited by 163
- 2025Reasoning Models Know When They're Right: Probing Hidden States for Self-Verificationcited by 159
- 2025The Danger of Overthinking: Examining the Reasoning-Action Dilemma in Agentic Taskscited by 152
- 2025LlamaV-o1: Rethinking Step-by-step Visual Reasoning in LLMscited by 147
- 2025AbstentionBench: Reasoning LLMs Fail on Unanswerable Questionscited by 145
- 2024O1 Replication Journey: A Strategic Progress Report -- Part 1cited by 143
- 2024Improve Vision Language Model Chain-of-thought Reasoningcited by 136
50 most cited of 166 All 166 (JSON)
Built on by
Citing papers per year: 2025: 7; 2026: 18.
25 papers.
Sort:
By area:
- 2025Are Your Agents Upward Deceivers?cited by 16
- 2025DiffuGuard: How Intrinsic Safety is Lost and Found in Diffusion Large Language Modelscited by 12
- 2026What Matters For Safety Alignment?cited by 7
- 2026Auditing Reasoning-Trace Memorization Claims after Unlearning with Head-Conditioned Canariescited by 7
- 2025Strong but Brittle: Simple Attacks Subvert Reasoning-based Safety Guardrailscited by 5
- 2025MultiPriv: Benchmarking Individual-Level Privacy Reasoning in Vision-Language Modelscited by 5
- 2025SafeRBench: Dissecting the Reasoning Safety of Large Language Modelscited by 4
- 2026FaithRL: Learning to Reason Faithfully through Step-Level Faithfulness Maximizationcited by 4
- 2025Critical or Compliant? The Double-Edged Sword of Reasoning in Chain-of-Thought Explanationscited by 3
- 2026Interpreting and Controlling LLM Reasoning through Integrated Policy Gradientcited by 3
- 2026Stop Tracking Me! Proactive Defense Against Attribute Inference Attack in LLMscited by 3
- 2026From P(y|x) to P(y): Investigating Reinforcement Learning in Pre-train Spacecited by 3
13 more
- 2025Rethinking Reasoning: A Survey on Reasoning-based Backdoors in LLMscited by 2
- 2026Pause or Fabricate? Training Language Models for Grounded Reasoningcited by 2
- 2026MM-THEBench: Do Reasoning MLLMs Think Reasonably?cited by 1
- 2026Are GUI Agents Focused Enough? Automated Distraction via Semantic-level UI Element Injectioncited by 1
- 2026ReasoningLens: Hierarchical Visualization and Diagnostic Auditing for Large Reasoning Modelscited by 1
- 2026EvalQReason: A Framework for Step-Level Reasoning Evaluation in Large Language Modelscited by 0
- 2026Answering the Wrong Question: Reasoning Trace Inversion for Abstention in LLMscited by 0
- 2026Strengthening Human-Centric Chain-of-Thought Reasoning Integrity in LLMs via a Structured Prompt Frameworkcited by 0
- 2026Does Reasoning Preserve Alignment? On the Trustworthiness of Large Reasoning Modelscited by 0
- 2026Scope3Trace: Evidence-Based Identification and Extraction of Scope 3 GHG Emissions from Sustainability Reportscited by 0
- 2026Risky Business: Measuring The Faithfulness-Safety Tensioncited by 0
- 2026Hallucination Across the Reasoning Lifecycle: Interface Visibility, Causal Evidence, and Release Control in Large Reasoning Modelscited by 0
- 2026Facts Don't Speak Louder than Words: The Behavioral Essence of Long-CoT Distillationcited by 0
- 2026Facts Don't Speak Louder than Words: The Behavioral Essence of Long-CoT Distillation13 Oct 2026
- 2026Hallucination Across the Reasoning Lifecycle: Interface Visibility, Causal Evidence, and Release Control in Large Reasoning Models4 Oct 2026
- 2026Risky Business: Measuring The Faithfulness-Safety Tension4 Aug 2026
- 2026Scope3Trace: Evidence-Based Identification and Extraction of Scope 3 GHG Emissions from Sustainability Reports19 Jul 2026
- 2026ReasoningLens: Hierarchical Visualization and Diagnostic Auditing for Large Reasoning Models22 Jun 2026
- 2026Does Reasoning Preserve Alignment? On the Trustworthiness of Large Reasoning Models9 Jun 2026
- 2026Auditing Reasoning-Trace Memorization Claims after Unlearning with Head-Conditioned Canaries17 May 2026
- 2026Pause or Fabricate? Training Language Models for Grounded Reasoning21 Apr 2026
- 2026From P(y|x) to P(y): Investigating Reinforcement Learning in Pre-train Space15 Apr 2026
- 2026Are GUI Agents Focused Enough? Automated Distraction via Semantic-level UI Element Injection9 Apr 2026
- 2026Strengthening Human-Centric Chain-of-Thought Reasoning Integrity in LLMs via a Structured Prompt Framework6 Apr 2026
- 2026Answering the Wrong Question: Reasoning Trace Inversion for Abstention in LLMs2 Apr 2026
13 more
- 2026Stop Tracking Me! Proactive Defense Against Attribute Inference Attack in LLMs12 Feb 2026
- 2026FaithRL: Learning to Reason Faithfully through Step-Level Faithfulness Maximization3 Feb 2026
- 2026Interpreting and Controlling LLM Reasoning through Integrated Policy Gradient2 Feb 2026
- 2026EvalQReason: A Framework for Step-Level Reasoning Evaluation in Large Language Models2 Feb 2026
- 2026MM-THEBench: Do Reasoning MLLMs Think Reasonably?30 Jan 2026
- 2026What Matters For Safety Alignment?7 Jan 2026
- 2025Are Your Agents Upward Deceivers?4 Dec 2025
- 2025MultiPriv: Benchmarking Individual-Level Privacy Reasoning in Vision-Language Models21 Nov 2025
- 2025SafeRBench: Dissecting the Reasoning Safety of Large Language Models19 Nov 2025
- 2025Critical or Compliant? The Double-Edged Sword of Reasoning in Chain-of-Thought Explanations15 Nov 2025
- 2025Strong but Brittle: Simple Attacks Subvert Reasoning-based Safety Guardrails13 Oct 2025
- 2025Rethinking Reasoning: A Survey on Reasoning-based Backdoors in LLMs9 Oct 2025
- 2025DiffuGuard: How Intrinsic Safety is Lost and Found in Diffusion Large Language Models29 Sep 2025
- 2026Auditing Reasoning-Trace Memorization Claims after Unlearning with Head-Conditioned Canariescited by 7
- 2025Strong but Brittle: Simple Attacks Subvert Reasoning-based Safety Guardrailscited by 5
- 2025MultiPriv: Benchmarking Individual-Level Privacy Reasoning in Vision-Language Modelscited by 5
- 2025SafeRBench: Dissecting the Reasoning Safety of Large Language Modelscited by 4
- 2026FaithRL: Learning to Reason Faithfully through Step-Level Faithfulness Maximizationcited by 4
- 2025Critical or Compliant? The Double-Edged Sword of Reasoning in Chain-of-Thought Explanationscited by 3
- 2026Interpreting and Controlling LLM Reasoning through Integrated Policy Gradientcited by 3
- 2026From P(y|x) to P(y): Investigating Reinforcement Learning in Pre-train Spacecited by 3
- 2025Rethinking Reasoning: A Survey on Reasoning-based Backdoors in LLMscited by 2
- 2026Pause or Fabricate? Training Language Models for Grounded Reasoningcited by 2
- 2026MM-THEBench: Do Reasoning MLLMs Think Reasonably?cited by 1
- 2026ReasoningLens: Hierarchical Visualization and Diagnostic Auditing for Large Reasoning Modelscited by 1
12 most cited of 19 All 19 (JSON)
- 2026Facts Don't Speak Louder than Words: The Behavioral Essence of Long-CoT Distillation13 Oct 2026
- 2026Hallucination Across the Reasoning Lifecycle: Interface Visibility, Causal Evidence, and Release Control in Large Reasoning Models4 Oct 2026
- 2026Risky Business: Measuring The Faithfulness-Safety Tension4 Aug 2026
- 2026ReasoningLens: Hierarchical Visualization and Diagnostic Auditing for Large Reasoning Models22 Jun 2026
- 2026Does Reasoning Preserve Alignment? On the Trustworthiness of Large Reasoning Models9 Jun 2026
- 2026Auditing Reasoning-Trace Memorization Claims after Unlearning with Head-Conditioned Canaries17 May 2026
- 2026Pause or Fabricate? Training Language Models for Grounded Reasoning21 Apr 2026
- 2026From P(y|x) to P(y): Investigating Reinforcement Learning in Pre-train Space15 Apr 2026
- 2026Strengthening Human-Centric Chain-of-Thought Reasoning Integrity in LLMs via a Structured Prompt Framework6 Apr 2026
- 2026Answering the Wrong Question: Reasoning Trace Inversion for Abstention in LLMs2 Apr 2026
- 2026FaithRL: Learning to Reason Faithfully through Step-Level Faithfulness Maximization3 Feb 2026
- 2026Interpreting and Controlling LLM Reasoning through Integrated Policy Gradient2 Feb 2026
12 newest of 19 All 19 (JSON)
- 2026What Matters For Safety Alignment?cited by 7
- 2026What Matters For Safety Alignment?7 Jan 2026
Similar, not cited
Close in wording, no citation either way
- 2025Red Teaming Large Reasoning Models
- 2026Shorter, but Still Trustworthy? An Empirical Study of Chain-of-Thought Compression
- 2025Investigating CoT Monitorability in Large Reasoning Models
- 2025ReFIne: A Framework for Trustworthy Large Reasoning Models with Reliability, Faithfulness, and Interpretability
- 2026Beyond Content Safety: Real-Time Monitoring for Reasoning Vulnerabilities in Large Language Models
- 2025Can Pruning Improve Reasoning? Revisiting Long-CoT Compression with Capability in Mind for Better Reasoning
- 2026Structural Anchors and Reasoning Fragility:Understanding CoT Robustness in LLM4Code
- 2025The CoT Encyclopedia: Analyzing, Predicting, and Controlling how a Reasoning Model will Think