mlsanityData to 14 Oct 2026

HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMs

arXiv 2412.18925 ↗First posted 25 Dec 2024cs.CL cs.AI cs.LGcited by 265

On the map: LLM reasoning → Clinical and biomedical applications

Abstract (arXiv)

The breakthrough of OpenAI o1 highlights the potential of enhancing reasoning to improve LLM. Yet, most research in reasoning has focused on mathematical tasks, leaving domains like medicine underexplored. The medical domain, though distinct from mathematics, also demands robust reasoning to provide reliable answers, given the high standards of healthcare. However, verifying medical reasoning is challenging, unlike those in mathematics. To address this, we propose verifiable medical problems with a medical verifier to check the correctness of model outputs. This verifiable nature enables advancements in medical reasoning through a two-stage approach: (1) using the verifier to guide the search for a complex reasoning trajectory for fine-tuning LLMs, (2) applying reinforcement learning (RL) with verifier-based rewards to enhance complex reasoning further. Finally, we introduce HuatuoGPT-o1, a medical LLM capable of complex reasoning, which outperforms general and medical-specific baselines using only 40K verifiable problems. Experiments show complex reasoning improves medical problem-solving and benefits more from RL. We hope our approach inspires advancements in reasoning across medical and other specialized domains.

Key facts. Cited by 265 papers in the map (data through 14 Oct 2026). Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning. Builds on 71 of its 82 references that are in the map. First posted on arXiv 25 Dec 2024.

Builds on

71 of 82 references.

References · 71

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  6. 2023Let's Verify Step by Stepcited by 3,789
  7. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  8. 2023Mistral 7Bcited by 3,164
  9. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  10. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  11. 2024Qwen2 Technical Reportcited by 2,120
  12. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
38 more
  1. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  2. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  3. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,434
  4. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,356
  5. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,288
  6. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 1,017
  7. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 940
  8. 2024Yi: Open Foundation Models by 01.AIcited by 719
  9. 2022Large Language Models Can Self-Improvecited by 664
  10. 2024DeepSeek LLM: Scaling Open-Source Language Models with Longtermismcited by 642
  11. 2023WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instructcited by 625
  12. 2022MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answeringcited by 543
  13. 2024LLaVA-CoT: Let Vision Language Models Reason Step-by-Stepcited by 454
  14. 2023CMMLU: Measuring massive multitask language understanding in Chinesecited by 449
  15. 2024Training Language Models to Self-Correct via Reinforcement Learningcited by 426
  16. 2023Towards Expert-Level Medical Question Answering with Large Language Modelscited by 384
  17. 2023The Curse of Recursion: Training on Generated Data Makes Models Forgetcited by 356
  18. 2023Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicinecited by 329
  19. 2024Deliberative Alignment: Reasoning Enables Safer Language Modelscited by 308
  20. 2024BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domainscited by 301
  21. 2023MedAlpaca -- An Open-Source Collection of Medical Conversational AI Models and Training Datacited by 286
  22. 2022Generating Sequences by Learning to Self-Correctcited by 277
  23. 2023MEDITRON-70B: Scaling Medical Pretraining for Large Language Modelscited by 274
  24. 2023Self-Consuming Generative Models Go MADcited by 250
  25. 2024ReFT: Reasoning with Reinforced Fine-Tuningcited by 239
  26. 2024Capabilities of Gemini Models in Medicinecited by 237
  27. 2024Agent Hospital: A Simulacrum of Hospital with Evolvable Medical Agentscited by 218
  28. 2023REFINER: Reasoning Feedback on Intermediate Representationscited by 205
  29. 2024HuatuoGPT-Vision, Towards Injecting Medical Visual Knowledge into Multimodal LLMs at Scalecited by 200
  30. 2018Learning by Playing - Solving Sparse Reward Tasks from Scratchcited by 167
  31. 2023Solving Challenging Math Word Problems Using GPT-4 Code Interpreter with Code-based Self-Verificationcited by 166
  32. 2022A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunitiescited by 163
  33. 2024Stream of Search (SoS): Learning to Search in Languagecited by 145
  34. 2024Marco-o1: Towards Open Reasoning Models for Open-Ended Solutionscited by 144
  35. 2024O1 Replication Journey: A Strategic Progress Report -- Part 1cited by 143
  36. 2024Evaluation of OpenAI o1: Opportunities and Challenges of AGIcited by 127
  37. 2023CMB: A Comprehensive Medical Benchmark in Chinesecited by 115
  38. 2024Pride and Prejudice: LLM Amplifies Self-Bias in Self-Refinementcited by 114

50 most cited of 71 All 71 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 153; 2026: 112.

265 papers.

Sort:

By area:

  1. 2025Search-o1: Agentic Search-Enhanced Large Reasoning Modelscited by 433
  2. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 384
  3. 2025Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domainscited by 366
  4. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 320
  5. 2026Self-Distillation Enables Continual Learningcited by 247
  6. 2025Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoningcited by 230
  7. 2025MedXpertQA: Benchmarking Expert-Level Medical Reasoning and Understandingcited by 224
  8. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 205
  9. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 180
  10. 2025Crossing the Reward Bridge: Expanding RL with Verifiable Rewards Across Diverse Domainscited by 156
  11. 2025OpenRubrics: Towards Scalable Synthetic Rubric Generation for Reward Modeling and LLM Alignmentcited by 132
  12. 2025MedReason: Eliciting Factual Medical Reasoning Steps in LLMs via Knowledge Graphscited by 110
38 more
  1. 2025JudgeLRM: Large Reasoning Models as a Judgecited by 78
  2. 2025From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discoverycited by 77
  3. 2025RedStar: Does Scaling Long-CoT Data Unlock Better Slow-Reasoning Systems?cited by 60
  4. 2025Baichuan-M2: Scaling Medical Capability with Large Verifier Systemcited by 59
  5. 2025Look-Back: Implicit Visual Re-focusing in MLLM Reasoningcited by 55
  6. 2025Agentic Reward Modeling: Integrating Human Preferences with Verifiable Correctness Signals for Reliable Reward Systemscited by 51
  7. 2025A Survey on the Optimization of Large Language Model-based Agentscited by 46
  8. 2025m1: Unleash the Potential of Test-Time Scaling for Medical Reasoning with Large Language Modelscited by 46
  9. 2025VerIF: Verification Engineering for Reinforcement Learning in Instruction Followingcited by 44
  10. 2025Baichuan-M1: Pushing the Medical Capability of Large Language Modelscited by 43
  11. 2025A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontierscited by 37
  12. 2025MedCaseReasoning: Evaluating and learning diagnostic reasoning from clinical case reportscited by 36
  13. 2025Beyond Distillation: Pushing the Limits of Medical LLM Reasoning with Minimalist Rule-Based RLcited by 36
  14. 2025Fino1: On the Transferability of Reasoning-Enhanced LLMs and Reinforcement Learning to Financecited by 35
  15. 2025LLMEval-Med: A Real-world Clinical Benchmark for Medical LLMs with Physician Validationcited by 34
  16. 2025ChestX-Reasoner: Advancing Radiology Foundation Models with Reasoning through Step-by-Step Verificationcited by 31
  17. 2025X-Reasoner: Towards Generalizable Reasoning Across Modalities and Domainscited by 31
  18. 2025Real-World Doctor Agent with Proactive Consultation through Multi-Agent Reinforcement Learningcited by 27
  19. 2025ReasonMed: A 370K Multi-Agent Generated Dataset for Advancing Medical Reasoningcited by 27
  20. 2025MedVLThinker: Simple Baselines for Multimodal Medical Reasoningcited by 27
  21. 2025MedS3: Towards Medical Slow Thinking with Self-Evolved Soft Dual-sided Process Supervisioncited by 24
  22. 2025Med-PRM: Medical Reasoning Models with Stepwise, Guideline-verified Process Rewardscited by 24
  23. 2025MedAgentGym: A Scalable Agentic Training Environment for Code-Centric Reasoning in Biomedical Data Sciencecited by 23
  24. 2025A Survey on Agentic Multimodal Large Language Modelscited by 23
  25. 2025Automating Expert-Level Medical Reasoning Evaluation of Large Language Modelscited by 22
  26. 2025DiagnosisArena: Benchmarking Diagnostic Reasoning for Large Language Modelscited by 21
  27. 2025Psyche-R1: Towards Reliable Psychological LLMs through Unified Empathy, Expertise, and Reasoningcited by 20
  28. 2025FineMedLM-o1: Enhancing Medical Knowledge Reasoning Ability of LLM from Supervised Fine-Tuning to Test-Time Trainingcited by 19
  29. 2025A Survey of Slow Thinking-based Reasoning LLMs using Reinforced Learning and Inference-time Scaling Lawcited by 19
  30. 2026EHRWorld: A Patient-Centric Medical World Model for Long-Horizon Clinical Trajectoriescited by 19
  31. 2025Disentangling Reasoning and Knowledge in Medical Large Language Modelscited by 18
  32. 2025ThinkPatterns-21k: A Systematic Study on the Impact of Thinking Patterns in LLMscited by 16
  33. 2025Reasoning Beyond Limits: Advances and Open Problems for LLMscited by 16
  34. 2026Entropy-Adaptive Fine-Tuning: Resolving Confident Conflicts to Mitigate Forgettingcited by 16
  35. 2025Generative AI Act II: Test Time Scaling Drives Cognition Engineeringcited by 15
  36. 2026Counteraction-Aware Multi-Teacher On-Policy Distillation for General Capability Recovery with Domain Preservationcited by 15
  37. 2025CoIn: Counting the Invisible Reasoning Tokens in Commercial Opaque LLM APIscited by 14
  38. 2025Knowledge or Reasoning? A Close Look at How LLMs Think Across Domainscited by 14

50 most cited of 265 All 265 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Reasoning LLMs in the Medical Domain: A Literature Survey
  2. 2025Fleming-R1: Toward Expert-Level Medical Reasoning via Reinforcement Learning
  3. 2024Critique of Impure Reason: Unveiling the reasoning behaviour of medical Large Language Models
  4. 2026Advances in LLM Reasoning Enable Flexibility in Clinical Problem-Solving
  5. 2025GMAI-VL-R1: Harnessing Reinforcement Learning for Multimodal Medical Reasoning
  6. 2026MedZERO: Self-Evolving Agents for Open-Ended Medical Reasoning Through Controlled Knowledge Accumulation
  7. 2025Med-RLVR: Emerging Medical Reasoning from a 3B base model via reinforcement Learning
  8. 2025Haibu Mathematical-Medical Intelligent Agent:Enhancing Large Language Model Reliability in Medical Tasks via Verifiable Reasoning Chains