mlsanityData to 7 Oct 2026

Chain-of-Reasoning: Towards Unified Mathematical Reasoning in Large Language Models via a Multi-Paradigm Perspective

arXiv 2501.11110 ↗First posted 19 Jan 2025cs.CLcited by 29

On the map: LLM reasoning → Mathematical reasoning

Abstract (arXiv)

Large Language Models (LLMs) have made notable progress in mathematical reasoning, yet often rely on single-paradigm reasoning, limiting their effectiveness across diverse tasks. We introduce Chain-of-Reasoning (CoR), a novel unified framework integrating multiple reasoning paradigms--Natural Language Reasoning (NLR), Algorithmic Reasoning (AR), and Symbolic Reasoning (SR)--to enable synergistic collaboration. CoR generates multiple potential answers via different reasoning paradigms and synthesizes them into a coherent final solution. We propose a Progressive Paradigm Training (PPT) strategy for models to progressively master these paradigms, leading to CoR-Math-7B. Experimental results demonstrate that CoR-Math-7B significantly outperforms current SOTA models, achieving up to a 41.0% absolute improvement over GPT-4o in theorem proving and a 15.0% improvement over RL-based methods on the MATH benchmark in arithmetic tasks. These results show the enhanced mathematical comprehension ability of our model, enabling zero-shot generalization across tasks.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 40 of its 52 references that are in the map. First posted on arXiv 19 Jan 2025.

Builds on

40 of 52 references.

Closest ancestors · 11

Same direction, newest first

  1. 2024Lean-STaR: Learning to Interleave Thinking and Proving14 Jul
  2. 2024DART-Math: Difficulty-Aware Rejection Tuning for Mathematical Problem-Solving18 Jun
  3. 2024Common 7B Language Models Already Possess Strong Math Capabilities7 Mar
  4. 2024MUSTARD: Mastering Uniform Synthesis of Theorem and Proof Data14 Feb
  5. 2024InternLM-Math: Open Math Large Language Models Toward Verifiable Reasoning9 Feb
  6. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models5 Feb
  7. 2023Llemma: An Open Language Model For Mathematics16 Oct
  8. 2023ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving29 Sep
  9. 2023WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct18 Aug
  10. 2022Solving Math Word Problems via Cooperative Reasoning induced Language Models28 Oct
  11. 2020Generative Language Modeling for Automated Theorem Proving7 Sep

Other references · 29

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  7. 2024GPT-4o System Cardcited by 4,467
  8. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  9. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  10. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  11. 2024Mixtral of Expertscited by 1,785
  12. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
17 more
  1. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,278
  2. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  3. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,057
  4. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 873
  5. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 724
  6. 2022PAL: Program-aided Language Modelscited by 635
  7. 2021MiniF2F: a cross-system benchmark for formal Olympiad-level mathematicscited by 380
  8. 2022HyperTree Proof Search for Neural Theorem Provingcited by 192
  9. 2024DeepSeek-Prover-V1.5: Harnessing Proof Assistant Feedback for Reinforcement Learning and Monte-Carlo Tree Searchcited by 189
  10. 2024Lean Workbook: A large-scale Lean problem set formalized from natural language math problemscited by 115
  11. 2023Algorithm of Thoughts: Enhancing Exploration of Ideas in Large Language Modelscited by 92
  12. 2025Unlocking Multimodal Mathematical Reasoning via Process Reward Modelcited by 64
  13. 2024TreeBoN: Enhancing Inference-Time Alignment with Speculative Tree-Search and Best-of-N Samplingcited by 61
  14. 2023LLMSTEP: LLM proofstep suggestions in Leancited by 32
  15. 2024DySpec: Faster Speculative Decoding with Dynamic Token Tree Structurecited by 25
  16. 2024Gender Bias in LLM-generated Interview Responsescited by 10
  17. 2023EALM: Introducing Multidimensional Ethical Alignment in Conversational Information Retrievalcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 16; 2026: 13.

29 papers.

Sort:

By area:

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  2. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 319
  3. 2025Beyond Pass@1: Self-Play with Variational Problem Synthesis Sustains RLVRcited by 53
  4. 2025SwS: Self-aware Weakness-driven Problem Synthesis in Reinforcement Learning for LLM Reasoningcited by 39
  5. 2025TreeReview: A Dynamic Tree of Questions Framework for Deep and Efficient LLM-based Scientific Peer Reviewcited by 31
  6. 2025DRP: Distilled Reasoning Pruning with Skill-aware Step Decomposition for Efficient Large Reasoning Modelscited by 29
  7. 2026Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Modelscited by 22
  8. 2025Learning to Reason via Mixture-of-Thought for Logical Reasoningcited by 20
  9. 2026AutoSOTA: An End-to-End Automated Research System for State-of-the-Art AI Model Discoverycited by 17
  10. 2025TL;DR: Too Long, Do Re-weighting for Efficient LLM Reasoning Compressioncited by 16
  11. 2025Learning How to Use Tools, Not Just When: Pattern-Aware Tool-Integrated Reasoningcited by 15
  12. 2025Find Your Optimal Teacher: Personalized Data Synthesis via Router-Guided Multi-Teacher Distillationcited by 15
17 more
  1. 2026ToolPRMBench: Evaluating and Advancing Process Reward Models for Tool-using Agentscited by 13
  2. 2025The Quest for Efficient Reasoning: A Data-Centric Benchmark to CoT Distillationcited by 7
  3. 2026Reward Modeling from Natural Language Human Feedbackcited by 7
  4. 2025LLMs as World Models: Data-Driven and Human-Centered Pre-Event Simulation for Disaster Impact Assessmentcited by 6
  5. 2025GuiLoMo: Allocating Expert Number and Rank for LoRA-MoE via Bilevel Optimization with GuidedSelection Vectorscited by 6
  6. 2025Training Large Language Models To Reason In Parallel With Global Forking Tokenscited by 6
  7. 2026Beyond Outliers: A Data-Free Layer-wise Mixed-Precision Quantization Approach Driven by Numerical and Structural Dual-Sensitivitycited by 5
  8. 2025Putting on the Thinking Hats: A Survey on Chain of Thought Fine-tuning from the Perspective of Human Reasoning Mechanismcited by 4
  9. 2026Training LLMs for Divide-and-Conquer Reasoning Elevates Test-Time Scalabilitycited by 4
  10. 2026Unified Data Selection for LLM Reasoningcited by 2
  11. 2025From Perception to Reasoning: Deep Thinking Empowers Multimodal Large Language Modelscited by 1
  12. 2026From Solvers to Research: Large Language Model-Driven Formal Mathematics at the Research Frontiercited by 1
  13. 2026Strategy Executability in Mathematical Reasoning: Leveraging Human-Model Differences for Effective Guidancecited by 0
  14. 2026Hallucination Self-Play: Bootstrapping Reinforced Detector via Evolved Generatorcited by 0
  15. 2026When Tokenization is Secretly Output Supervisioncited by 0
  16. 2026Efficient Reasoning Exploration via State-Conditioned Latent Steering with Progress Guidancecited by 0
  17. 2026MetaCtrl: Your Large Language Models Can Reason Better and More Concisely with a Metacognitive Controllercited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A Survey on Large Language Models for Mathematical Reasoning
  2. 2026Mathematical Reasoning in Large Language Models: Benchmarks, Architectures, Evaluation, and Open Challenges
  3. 2024CoMAT: Chain of Mathematically Annotated Thought Improves Mathematical Reasoning
  4. 2025Premise-Augmented Reasoning Chains Improve Error Identification in Math reasoning with LLMs
  5. 2025Phi-4-Mini-Reasoning: Exploring the Limits of Small Reasoning Language Models in Math
  6. 2023Learning Multi-Step Reasoning by Solving Arithmetic Tasks
  7. 2025MathFimer: Enhancing Mathematical Reasoning by Expanding Reasoning Steps through Fill-in-the-Middle Task
  8. 2024Small Language Models are Equation Reasoners