Reasoning Fails Where Step Flow Breaks
On the map: LLM reasoning → Test-time compute and efficient thinking
Abstract (arXiv)
Large reasoning models (LRMs) that generate long chains of thought now perform well on multi-step math, science, and coding tasks. However, their behavior is still unstable and hard to interpret, and existing analysis tools struggle with such long, structured reasoning traces. We introduce Step-Saliency, which pools attention--gradient scores into step-to-step maps along the question--thinking--summary trajectory. Across several models, Step-Saliency reveals two recurring information-flow failures: Shallow Lock-in, where shallow layers over-focus on the current step and barely use earlier context, and Deep Decay, where deep layers gradually lose saliency on the thinking segment and the summary increasingly attends to itself and the last few steps. Motivated by these patterns, we propose StepFlow, a saliency-inspired test-time intervention that adjusts shallow saliency patterns measured by Step-Saliency via Odds-Equal Bridge and adds a small step-level residual in deep layers via Step Momentum Injection. StepFlow improves accuracy on math, science, and coding tasks across multiple LRMs without retraining, indicating that repairing information flow can recover part of their missing reasoning performance.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 36 of its 59 references that are in the map. First posted on arXiv 8 Apr 2026.
Builds on
36 of 59 references.
Closest ancestors · 1
Same direction, newest first
Other references · 35
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
- 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
- 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
- 2019Attention is not Explanationcited by 929
- 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 703
- 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 560
- 2025Reasoning Models Don't Always Say What They Thinkcited by 415
- 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 384
- 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 320
- 2025Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Modelscited by 293
- 2024Large Language Models for Mathematical Reasoning: Progresses and Challengescited by 284
- 2019Analyzing the Structure of Attention in a Transformer Language Modelcited by 243
23 more
- 2020Probing Pretrained Language Models for Lexical Semanticscited by 156
- 2024Making Reasoning Matter: Measuring and Improving Faithfulness of Chain-of-Thought Reasoningcited by 110
- 2023Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMscited by 91
- 2019Attention Interpretability Across NLP Taskscited by 71
- 2025Evaluating Step-by-step Reasoning Traces: A Surveycited by 63
- 2017Towards better understanding of gradient-based attribution methods for Deep Neural Networkscited by 54
- 2024How to think step-by-step: A mechanistic understanding of chain-of-thought reasoningcited by 53
- 2024From Understanding to Utilization: A Survey on Explainability for Large Language Modelscited by 48
- 2023A Survey on Long Text Modeling with Transformerscited by 47
- 2020Gradient-based Analysis of NLP Models is Manipulablecited by 41
- 2021Evaluating Saliency Methods for Neural Language Modelscited by 37
- 2023Analyzing Chain-of-Thought Prompting in Large Language Models via Gradient-based Feature Attributionscited by 32
- 2024XAI meets LLMs: A Survey of the Relation between Explainable AI and Large Language Modelscited by 23
- 2025Large Language Models and Mathematical Reasoning Failurescited by 22
- 2020Cluster-Former: Clustering-based Sparse Transformer for Long-Range Dependency Encodingcited by 19
- 2025Large Language Models Encode Semantics and Alignment in Linearly Separable Representationscited by 17
- 2025Knowledge or Reasoning? A Close Look at How LLMs Think Across Domainscited by 14
- 2025Improving Complex Reasoning with Dynamic Prompt Corruption: A soft prompt Optimization Approachcited by 9
- 2025Towards Advanced Mathematical Reasoning for LLMs via First-Order Logic Theorem Provingcited by 8
- 2025Don't Take Things Out of Context: Attention Intervention for Enhancing Chain-of-Thought Reasoning in Large Language Modelscited by 7
- 2020Attention is Not Only a Weight: Analyzing Transformers with Vector Normscited by 6
- 2025Demystifying Scientific Problem-Solving in LLMs by Probing Knowledge and Reasoningcited by 4
- 2023Bi-directional personalization reinforcement learning-based architecture with active learning using a multi-model data service for the travel nursing industrycited by 1
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025Making Slow Thinking Faster: Compressing LLM Chain-of-Thought via Step Entropy
- 2026CRISP: Compressing Redundancy in Chain-of-Thought via Intrinsic Saliency Pruning
- 2025The Evolution of Thought: Tracking LLM Overthinking via Reasoning Dynamics Analysis
- 2026How Hard Does It Think? Analyzing Step-Aware Reasoning Energy in LLM Chain-of-Thought Trajectories
- 2026Saliency-R1: Enforcing Interpretable and Faithful Vision-language Reasoning via Saliency-map Alignment Reward
- 2026Test-time Scaling over Perception: Resolving the Grounding Paradox in Thinking with Images
- 2025When Can Large Reasoning Models Save Thinking? Mechanistic Analysis of Behavioral Divergence in Reasoning
- 2025Tracing the Traces: Latent Temporal Signals for Efficient and Accurate Reasoning