Towards Stepwise Domain Knowledge-Driven Reasoning Optimization and Reflection Improvement
On the map: LLM reasoning → Prompting and chain-of-thought reasoning
Abstract (arXiv)
Recently, stepwise supervision on Chain of Thoughts (CoTs) presents an enhancement on the logical reasoning tasks such as coding and math, with the help of Monte Carlo Tree Search (MCTS). However, its contribution to tasks requiring domain-specific expertise and knowledge remains unexplored. Motivated by the interest, we identify several potential challenges of vanilla MCTS within this context, and propose the framework of Stepwise Domain Knowledge-Driven Reasoning Optimization, employing the MCTS algorithm to develop step-level supervision for problems that require essential comprehension, reasoning, and specialized knowledge. Additionally, we also introduce the Preference Optimization towards Reflection Paths, which iteratively learns self-reflection on the reasoning thoughts from better perspectives. We have conducted extensive experiments to evaluate the advantage of the methodologies. Empirical results demonstrate the effectiveness on various legal-domain problems. We also report a diverse set of valuable findings, hoping to encourage the enthusiasm to the research of domain-specific LLMs and MCTS.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 16 of its 34 references that are in the map. First posted on arXiv 12 Apr 2025.
Builds on
16 of 34 references.
References · 16
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 734
- 2024ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Searchcited by 344
- 2024Smaug: Fixing Failure Modes of Preference Optimisation with DPO-Positivecited by 241
- 2024AlphaMath Almost Zero: Process Supervision without Processcited by 162
- 2024Accessing GPT-4 level Mathematical Olympiad Solutions via Monte Carlo Tree Self-refine with LLaMa-3 8Bcited by 158
- 2024O1 Replication Journey: A Strategic Progress Report -- Part 1cited by 143
- 2024Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizingcited by 125
- 2023DISC-LawLLM: Fine-tuning Large Language Models for Intelligent Legal Servicescited by 120
- 2024Towards Analyzing and Understanding the Limitations of DPO: A Theoretical Perspectivecited by 69
- 2024Interpretable Contrastive Monte Carlo Tree Search Reasoningcited by 64
- 2024Make Every Move Count: LLM-based High-Quality RTL Code Generation Using MCTScited by 43
4 more
- 2024Towards Self-Improvement of LLMs via MCTS: Leveraging Stepwise Knowledge with Curriculum Preference Learningcited by 30
- 2024Reverse Thinking Makes LLMs Stronger Reasonerscited by 29
- 2024RethinkMCTS: Refining Erroneous Thoughts in Monte Carlo Tree Search for Code Generationcited by 26
- 2024Can We Further Elicit Reasoning in LLMs? Critic-Guided Planning with Retrieval-Augmentation for Solving Challenging Taskscited by 22
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025Enhancing Reasoning through Process Supervision with Monte Carlo Tree Search
- 2024Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMs
- 2025From Reasoning to Super-Intelligence: A Search-Theoretic Perspective
- 2026Process Supervision for Chain-of-Thought Reasoning via Monte Carlo Net Information Gain
- 2025MITS: Enhanced Tree Search Reasoning for LLMs via Pointwise Mutual Information
- 2025Less is More Tokens: Efficient Math Reasoning via Difficulty-Aware Chain-of-Thought Distillation
- 2026Rewarding Novel Deductions: Solver-guided Process Supervision for Logical Reasoning
- 2026IsoSci: A Benchmark of Isomorphic Cross-Domain Science Problems for Evaluating Reasoning versus Knowledge Retrieval in LLMs