# ReasonMed: A 370K Multi-Agent Generated Dataset for Advancing Medical Reasoning

arXiv: https://arxiv.org/abs/2506.09513 · HTML: https://mlsanity.com/paper/2506.09513/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning.
- Builds on 19 of its 36 references that are in the map.
- First posted on arXiv 11 Jun 2025.

## Placed in

- LLM reasoning / [Clinical and biomedical applications](https://mlsanity.com/lr/clinical-and-biomedical-applications/)
- LLM agents and tool use / [Medical and scientific domain agents](https://mlsanity.com/agents/medical-and-scientific-domain-agents/)

## Builds on (most cited first, 19 of 19)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2308.00352/) (2023)
- [A Survey on LLM-as-a-Judge](https://mlsanity.com/paper/2411.15594/) (2024)
- [What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams](https://mlsanity.com/paper/2009.13081/) (2020)
- [s1: Simple test-time scaling](https://mlsanity.com/paper/2501.19393/) (2025)
- [Large Language Models for Mathematical Reasoning: Progresses and Challenges](https://mlsanity.com/paper/2402.00157/) (2024)
- [HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMs](https://mlsanity.com/paper/2412.18925/) (2024)
- [Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoning](https://mlsanity.com/paper/2506.07044/) (2025)
- [Can large language models reason about medical questions?](https://mlsanity.com/paper/2207.08143/) (2022)
- [O1 Replication Journey: A Strategic Progress Report -- Part 1](https://mlsanity.com/paper/2410.18982/) (2024)
- [Competitive Programming with Large Reasoning Models](https://mlsanity.com/paper/2502.06807/) (2025)
- [Logical Reasoning in Large Language Models: A Survey](https://mlsanity.com/paper/2502.09100/) (2025)
- [AgentCoder: Multi-Agent-based Code Generation with Iterative Testing and Optimisation](https://mlsanity.com/paper/2312.13010/) (2023)
- [BOLT: Bootstrap Long Chain-of-Thought in Language Models without Distillation](https://mlsanity.com/paper/2502.03860/) (2025)
- [Towards Leveraging Large Language Models for Automated Medical Q&A Evaluation](https://mlsanity.com/paper/2409.01941/) (2024)
- [Can Large Language Models Be Query Optimizer for Relational Databases?](https://mlsanity.com/paper/2502.05562/) (2025)
- [Adaptive Reasoning and Acting in Medical Language Agents](https://mlsanity.com/paper/2410.10020/) (2024)
- [DialogueAgents: A Hybrid Agent-Based Speech Synthesis Framework for Multi-Party Dialogue](https://mlsanity.com/paper/2504.14482/) (2025)

## Built on by (most cited first, 27 of 27)

- [A Survey of Reinforcement Learning for Large Reasoning Models](https://mlsanity.com/paper/2509.08827/) (2025)
- [A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontiers](https://mlsanity.com/paper/2508.21148/) (2025)
- [LegalDelta: Enhancing Legal Reasoning in LLMs via Reinforcement Learning with Chain-of-Thought Guided Information Gain](https://mlsanity.com/paper/2508.12281/) (2025)
- [InfiMed-ORBIT: Aligning LLMs on Open-Ended Complex Tasks via Rubric-Based Incremental Training](https://mlsanity.com/paper/2510.15859/) (2025)
- [S-Chain: Structured Visual Chain-of-Thought For Medicine](https://mlsanity.com/paper/2510.22728/) (2025)
- [OralGPT-Omni: A Versatile Dental Multimodal Large Language Model](https://mlsanity.com/paper/2511.22055/) (2025)
- [Medical Reasoning in the Era of LLMs: A Systematic Review of Enhancement Techniques and Applications](https://mlsanity.com/paper/2508.00669/) (2025)
- [OctoMed: Data Recipes for State-of-the-Art Multimodal Medical Reasoning](https://mlsanity.com/paper/2511.23269/) (2025)
- [Optimsyn: Influence-Guided Rubrics Optimization for Synthetic Data Generation](https://mlsanity.com/paper/2604.00536/) (2026)
- [ToolLibGen: Scalable Automatic Tool Creation and Aggregation for LLM Reasoning](https://mlsanity.com/paper/2510.07768/) (2025)
- [Can LLM Agents Generate Real-World Evidence? Evaluating Observational Studies in Medical Databases](https://mlsanity.com/paper/2603.22767/) (2026)
- [Towards Efficient Medical Reasoning with Minimal Fine-Tuning Data](https://mlsanity.com/paper/2508.01450/) (2025)
- [TumorChain: Interleaved Multimodal Chain-of-Thought Reasoning for Traceable Clinical Tumor Analysis](https://mlsanity.com/paper/2603.05867/) (2026)
- [Explore-Execute Chain: Towards an Efficient Structured Reasoning Paradigm](https://mlsanity.com/paper/2509.23946/) (2025)
- [Empowering Real-World: A Survey on the Technology, Practice, and Evaluation of LLM-driven Industry Agents](https://mlsanity.com/paper/2510.17491/) (2025)
- [Knowledge-Augmented Long-CoT Generation for Complex Biomolecular Reasoning](https://mlsanity.com/paper/2511.08024/) (2025)
- [Towards Trustworthy Dermatology MLLMs: A Benchmark and Multimodal Evaluator for Diagnostic Narratives](https://mlsanity.com/paper/2511.09195/) (2025)
- [Step-CoT: Stepwise Visual Chain-of-Thought for Medical Visual Question Answering](https://mlsanity.com/paper/2603.13878/) (2026)
- [EHRBench: An Automated and Reliable EHR-based Benchmark for Clinical Decision Making with LLMs](https://mlsanity.com/paper/2605.30637/) (2026)
- [Towards Next-Generation Healthcare: A Survey of Medical Embodied AI for Perception, Decision-Making, and Action](https://mlsanity.com/paper/2606.15647/) (2026)
- [A Causal Language Modeling Detour Improves Encoder Continued Pretraining](https://mlsanity.com/paper/2605.12438/) (2026)
- [Aligning Clinical Needs and AI Capabilities: A Survey on LLMs for Medical Reasoning](https://mlsanity.com/paper/2607.07761/) (2026)
- [HiMed: Incentivizing Hindi Reasoning in Medical LLMs](https://mlsanity.com/paper/2605.24635/) (2026)
- [eCREAM-MedCorpus A Large-Scale Corpus of Clinical Notes for Italian](https://mlsanity.com/paper/2606.12569/) (2026)
- [CARE: Causally-Aligned Reasoning Exploration for Medical Large Language Models](https://mlsanity.com/paper/2608.26147/) (2026)
- [InfiMed2: A Generalist Medical Multimodal Foundation Model from Contextual Evidence and Stability-Aware Supervision](https://mlsanity.com/paper/2609.34798/) (2026)
- [Taking the PULSE of Medical Reasoning: Step-Level Retrieval Timing for Knowledge-Grounded Inference](https://mlsanity.com/paper/2610.20292/) (2026)

## Abstract (arXiv)

Reasoning-based large language models have excelled in mathematics and programming, yet their potential in knowledge-intensive medical question answering remains underexplored and insufficiently validated in clinical contexts. To bridge this gap, we introduce ReasonMed, the largest medical reasoning dataset to date, comprising 370k high-quality examples distilled from 1.75 million initial reasoning paths generated by complementary LLMs and curated through a cost-efficient easy-medium-difficult (EMD) pipeline. ReasonMed is built through a multi-agent generation, verification, and refinement process, in which an Error Refiner improves reasoning paths by correcting error-prone steps identified by a verifier. Using ReasonMed, we investigate effective strategies for training medical reasoning models and find that integrating detailed CoT reasoning with concise answer summaries yields the most robust fine-tuning results. Models trained on ReasonMed set a new benchmark: ReasonMed-7B surpasses the prior best sub-10B models by 4.17% and even exceeds LLaMA3.1-70B on PubMedQA by 4.60%. When scaled to ReasonMed-14B, it remains highly competitive, underscoring consistent scaling potential. The codes and datasets are available at https://github.com/YuSun-Work/ReasonMed.