# What Defines Good Reasoning in LLMs? Dissecting Reasoning Steps with Multi-Aspect Evaluation

arXiv: https://arxiv.org/abs/2510.20603 · HTML: https://mlsanity.com/paper/2510.20603/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 12 of its 43 references that are in the map.
- First posted on arXiv 23 Oct 2025.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 12 of 12)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [s1: Simple test-time scaling](https://mlsanity.com/paper/2501.19393/) (2025)
- [Process Reinforcement through Implicit Rewards](https://mlsanity.com/paper/2502.01456/) (2025)
- [Improve Mathematical Reasoning in Language Models by Automated Process Supervision](https://mlsanity.com/paper/2406.06592/) (2024)
- [DeepSeek-Prover-V2: Advancing Formal Mathematical Reasoning via Reinforcement Learning for Subgoal Decomposition](https://mlsanity.com/paper/2504.21801/) (2025)
- [Common 7B Language Models Already Possess Strong Math Capabilities](https://mlsanity.com/paper/2403.04706/) (2024)
- [LLM Reasoners: New Evaluation, Library, and Analysis of Step-by-Step Reasoning with Large Language Models](https://mlsanity.com/paper/2404.05221/) (2024)
- [Self-rewarding correction for mathematical reasoning](https://mlsanity.com/paper/2502.19613/) (2025)
- [Evaluating Step-by-step Reasoning Traces: A Survey](https://mlsanity.com/paper/2502.12289/) (2025)
- [THINK-Bench: Evaluating Thinking Efficiency and Chain-of-Thought Quality of Large Reasoning Models](https://mlsanity.com/paper/2505.22113/) (2025)
- [CMCTS: A Constrained Monte Carlo Tree Search Framework for Mathematical Reasoning in Large Language Model](https://mlsanity.com/paper/2502.11169/) (2025)

## Built on by (most cited first, 10 of 10)

- [ATLAS: Verifier-Guided Adaptive Latent Activation Steering for Efficient LLM Reasoning](https://mlsanity.com/paper/2601.03093/) (2026)
- [What Makes Good Multilingual Reasoning? Disentangling Traces with Measurable Features](https://mlsanity.com/paper/2604.04720/) (2026)
- [The Moral Consistency Pipeline: Continuous Ethical Evaluation for Large Language Models](https://mlsanity.com/paper/2512.03026/) (2025)
- [InfoDensity: Rewarding Information-Dense Traces for Efficient Reasoning](https://mlsanity.com/paper/2603.17310/) (2026)
- [Decomposing the Delta: What Do Models Actually Learn from Preference Pairs?](https://mlsanity.com/paper/2604.08723/) (2026)
- [Beyond Output Correctness: Benchmarking and Evaluating Large Language Model Reasoning in Coding Tasks](https://mlsanity.com/paper/2604.12379/) (2026)
- [Reasoning Models Know What's Important, and Encode It in Their Activations](https://mlsanity.com/paper/2604.18307/) (2026)
- [Reasoning Denoiser: Denoising Reasoning Traces for Hallucination Detection in Large Reasoning Models](https://mlsanity.com/paper/2607.22098/) (2026)
- [EvalQReason: A Framework for Step-Level Reasoning Evaluation in Large Language Models](https://mlsanity.com/paper/2602.02295/) (2026)
- [Do LLMs Exhibit Coherent Knowledge Structures in Mathematical Reasoning? A Perspective from Knowledge Space Theory](https://mlsanity.com/paper/2609.05245/) (2026)

## Abstract (arXiv)

Evaluating large language models (LLMs) on final-answer correctness is the dominant paradigm. This approach, however, provides a coarse signal for model improvement and overlooks the quality of the underlying reasoning process. We argue that a more granular evaluation of reasoning offers a more effective path to building robust models. We decompose reasoning quality into two dimensions: relevance and coherence. Relevance measures if a step is grounded in the problem; coherence measures if it follows logically from prior steps. To measure these aspects reliably, we introduce causal stepwise evaluation (CaSE). This method assesses each reasoning step using only its preceding context, which avoids hindsight bias. We validate CaSE against human judgments on our new expert-annotated benchmarks, MRa-GSM8K and MRa-MATH. More importantly, we show that curating training data with CaSE-evaluated relevance and coherence directly improves final task performance. Our work provides a scalable framework for analyzing, debugging, and improving LLM reasoning, demonstrating the practical value of moving beyond validity checks.