mlsanityData to 7 Oct 2026

Towards A Unified View of Answer Calibration for Multi-Step Reasoning

arXiv 2311.09101 ↗First posted 15 Nov 2023cs.CL cs.AI cs.IRcited by 8

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Large Language Models (LLMs) employing Chain-of-Thought (CoT) prompting have broadened the scope for improving multi-step reasoning capabilities. We generally divide multi-step reasoning into two phases: path generation to generate the reasoning path(s); and answer calibration post-processing the reasoning path(s) to obtain a final answer. However, the existing literature lacks systematic analysis on different answer calibration approaches. In this paper, we summarize the taxonomy of recent answer calibration techniques and break them down into step-level and path-level strategies. We then conduct a thorough evaluation on these strategies from a unified view, systematically scrutinizing step-level and path-level answer calibration across multiple paths. Experimental results reveal that integrating the dominance of both strategies tends to derive optimal outcomes. Our study holds the potential to illuminate key insights for optimizing multi-step reasoning with answer calibration.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 27 of its 87 references that are in the map. First posted on arXiv 15 Nov 2023.

Builds on

27 of 87 references.

Closest ancestors · 10

Same direction, newest first

  1. 2023Resprompt: Residual Connection Prompting Advances Multi-Step Reasoning in Large Language Models7 Oct
  2. 2023Design of Chain-of-Thought in Math Problem Solving20 Sep
  3. 2023Knowledge-Driven CoT: Exploring Faithful Reasoning in LLMs for Knowledge-intensive Question Answering25 Aug
  4. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Models18 Aug
  5. 2023Thinking Like an Expert:Multimodal Hypergraph-of-Thought (HoT) Reasoning to boost Foundation Modals11 Aug
  6. 2023Boosting Language Models Reasoning with Chain-of-Knowledge Prompting10 Jun
  7. 2023Prompt Space Optimizing Few-shot Reasoning Success with Large Language Models6 Jun
  8. 2023Beyond Chain-of-Thought, Effective Graph-of-Thought Reasoning in Language Models26 May
  9. 2023Hint of Pseudo Code (HoPC): Zero-Shot Step by Step Pseudo Code Reasoning Prompting19 May
  10. 2023Progressive-Hint Prompting Improves Reasoning in Large Language Models19 Apr

Other references · 17

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  3. 2023A Survey of Large Language Modelscited by 2,537
  4. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  5. 2023FireAct: Toward Language Agent Fine-tuningcited by 247
  6. 2023Better Zero-Shot Reasoning with Role-Play Promptingcited by 220
  7. 2023Large Language Model Guided Tree-of-Thoughtcited by 218
  8. 2023Chain of Hindsight Aligns Language Models with Feedbackcited by 147
  9. 2023Deductive Verification of Chain-of-Thought Reasoningcited by 122
  10. 2023Algorithm of Thoughts: Enhancing Exploration of Ideas in Large Language Modelscited by 92
  11. 2023RCOT: Detecting and Rectifying Factual Inconsistency in Reasoning by Reversing Chain-of-Thoughtcited by 47
  12. 2023Forward-Backward Reasoning in Large Language Models for Mathematical Verificationcited by 39
5 more
  1. 2023Re-Reading Improves Reasoning in Large Language Modelscited by 37
  2. 2023Interactive Natural Language Processingcited by 31
  3. 2023GraphReason: Enhancing Reasoning Capabilities of Large Language Models through A Graph-Based Verification Approachcited by 21
  4. 2023When Do Program-of-Thoughts Work for Reasoning?cited by 19
  5. 2023Meta-Reasoning: Semantics-Symbol Deconstruction for Large Language Modelscited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 1; 2026: 5.

8 papers.

  1. 2024A Survey of Mathematical Reasoning in the Era of Multimodal Large Language Model: Benchmark, Method & Challengescited by 55
  2. 2026BAGEN: Are LLM Agents Budget-Aware?cited by 12
  3. 2025Long Context Scaling: Divide and Conquer via Multi-Agent Question-driven Collaborationcited by 7
  4. 2026Localizing Credit at the Divergence: Path-Conditioned Self-Distillation for LLM Reasoningcited by 4
  5. 2024Brain-Inspired Two-Stage Approach: Enhancing Mathematical Reasoning by Imitating Human Thought Processescited by 3
  6. 2026The Structural Attention Tax: How Retrieval Format Hijacks In-Context Learning Independent of Contentcited by 1
  7. 2026AutoCRAT: Within-trajectory Joint Control of Stochasticity and Compute for LLM Reasoningcited by 1
  8. 2026Answer Engineering: Local Trajectory Editing for Protocol-Constrained Decision Making in Large Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Early Stopping Chain-of-thoughts in Large Language Models
  2. 2024Can We Verify Step by Step for Incorrect Answer Detection?
  3. 2026Rethinking Reasoning Paths as Phase-Structured Trajectories
  4. 2023Graph Elicitation for Guiding Multi-Step Reasoning in Large Language Models
  5. 2024Chain-of-Probe: Examining the Necessity and Accuracy of CoT Step-by-Step
  6. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models
  7. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions
  8. 2024ProcBench: Benchmark for Multi-Step Reasoning and Following Procedure