Structural Anchors and Reasoning Fragility:Understanding CoT Robustness in LLM4Code
On the map: LLM reasoning → Prompting and chain-of-thought reasoning
Abstract (arXiv)
Chain-of-Thought (CoT) prompting is widely used to elicit explicit reasoning from large language models for code (LLM4Code). However, its impact on robustness and the stability of reasoning trajectories under realistic input perturbations remains poorly understood. Prior work has largely evaluated CoT through final correctness, leaving a critical gap in understanding how CoT reshapes internal uncertainty dynamics and why it sometimes harms rather than helps code generation. We suggest that CoT is not uniformly beneficial; instead, its robustness depends on whether perturbations destabilize structurally sensitive commitment points along the reasoning-to-code trajectory. We conduct a controlled, large-scale empirical study of CoT across six models and two code benchmarks (MHPP and BigCodeBench), subjecting task docstrings to systematic character-, word-, and sentence-level perturbations. We instrument full generation traces with token-level uncertainty and define three novel structural anchors: reasoning-code transition, symbolic commitment, and algorithmic articulation. Findings: (1) CoT does not yield uniform performance or robustness gains: its benefits are contingent on model family, task structure, and prompt explicitness. (2) CoT and No-CoT exhibit distinct robustness profiles, with different perturbation families triggering different failure modes. (3) We identify three recurrent trajectory deformations--Lengthening, Branching, and Simplification--that systematically emerge when perturbations interact with structural anchors and explain failure patterns. (4) Early-stage uncertainty serves as a reliable diagnostic signal for localizing where trajectory instability begins around sensitive anchors. These results provide a unified explanation for CoT's mixed performance and suggest design principles for building more robust reasoning-based code generators.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 20 of its 51 references that are in the map. First posted on arXiv 14 Apr 2026.
Builds on
20 of 51 references.
Closest ancestors · 2
Same direction, newest first
- 2023Towards Better Chain-of-Thought Prompting Strategies: A Survey8 Oct
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Models21 Mar
Other references · 18
- 2021Evaluating Large Language Models Trained on Codecited by 8,320
- 2021Program Synthesis with Large Language Modelscited by 3,637
- 2023Qwen Technical Reportcited by 3,344
- 2023Code Llama: Open Foundation Models for Codecited by 2,139
- 2022Language Models (Mostly) Know What They Knowcited by 1,856
- 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,181
- 2021Measuring Coding Challenge Competence With APPScited by 902
- 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 885
- 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 819
- 2020GraphCodeBERT: Pre-training Code Representations with Data Flowcited by 609
- 2024BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructionscited by 546
- 2023Chain of Code: Reasoning with a Language Model-Augmented Code Emulatorcited by 123
6 more
- 2022ReCode: Robustness Evaluation of Code Generation Modelscited by 103
- 2024Instruction Tuning for Secure Code Generationcited by 68
- 2025Uncertainty-Guided Chain-of-Thought for Code Generation with LLMscited by 32
- 2024MHPP: Exploring the Capabilities and Limitations of Language Models Beyond Basic Code Generationcited by 26
- 2024An Empirical Study on the Code Refactoring Capability of Large Language Modelscited by 19
- 2025THINK-Bench: Evaluating Thinking Efficiency and Chain-of-Thought Quality of Large Reasoning Modelscited by 19
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Fragile Thoughts: How Large Language Models Handle Chain-of-Thought Perturbations
- 2025Understanding Chain-of-Thought Effectiveness in Code Generation: An Empirical and Information-Theoretic Analysis
- 2024Deciphering the Factors Influencing the Efficacy of Chain-of-Thought: Probability, Memorization, and Noisy Reasoning
- 2025Chain-of-Thought Tokens are Computer Program Variables
- 2023Measuring Faithfulness in Chain-of-Thought Reasoning
- 2025The Curse of CoT: On the Limitations of Chain-of-Thought in In-Context Learning
- 2024To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning
- 2026How Far Ahead Do LLMs Plan? Uncovering the Latent Horizon in Chain-of-Thought Reasoning