mlsanityData to 14 Oct 2026

AnesSuite: A Comprehensive Benchmark and Dataset Suite for Anesthesiology Reasoning in LLMs

arXiv 2504.02404 ↗First posted 3 Apr 2025cs.CLcited by 1

On the map: LLM reasoning → Clinical and biomedical applications

Abstract (arXiv)

The application of large language models (LLMs) in the medical field has garnered significant attention, yet their reasoning capabilities in more specialized domains like anesthesiology remain underexplored. To bridge this gap, we introduce AnesSuite, the first comprehensive dataset suite specifically designed for anesthesiology reasoning in LLMs. The suite features AnesBench, an evaluation benchmark tailored to assess anesthesiology-related reasoning across three levels: factual retrieval (System 1), hybrid reasoning (System 1.x), and complex decision-making (System 2). Alongside this benchmark, the suite includes three training datasets that provide an infrastructure for continued pre-training (CPT), supervised fine-tuning (SFT), and reinforcement learning with verifiable rewards (RLVR). Leveraging this suite, we develop Morpheus, the first baseline model collection for anesthesiology reasoning. Despite undergoing limited training with SFT and group relative policy optimization (GRPO), Morpheus not only achieves substantial improvements in anesthesiology that rival larger-scale models, but also demonstrates enhanced reasoning capabilities across general medical and broad-domain benchmarks. Furthermore, through comprehensive evaluations and experiments, we analyze the key factors influencing anesthesiology reasoning performance, including model characteristics, training strategies and training data. Both AnesSuite and Morpheus will be open-sourced at https://github.com/MiliLab/AnesSuite.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning. Builds on 55 of its 114 references that are in the map. First posted on arXiv 3 Apr 2025.

Builds on

55 of 114 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Citrus: Leveraging Expert Cognitive Pathways in a Medical Language Model for Advanced Medical Decision Support25 Feb
  2. 2025O1 Replication Journey -- Part 3: Inference-time Scaling for Medical Reasoning11 Jan
  3. 2024HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMs25 Dec
  4. 2023Large Language Models Need Holistically Thought in Medical Conversational QA9 May

Other references · 51

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  6. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  7. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  8. 2024Qwen2.5 Technical Reportcited by 4,942
  9. 2023Let's Verify Step by Stepcited by 3,789
  10. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  11. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  12. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
38 more
  1. 2025Gemma 3 Technical Reportcited by 1,824
  2. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
  3. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,356
  4. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
  5. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  6. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,121
  7. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 1,053
  8. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 940
  9. 2024Yi: Open Foundation Models by 01.AIcited by 719
  10. 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 716
  11. 2024Phi-4 Technical Reportcited by 711
  12. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 708
  13. 2023Baichuan 2: Open Large-scale Language Modelscited by 670
  14. 2022MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answeringcited by 543
  15. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 500
  16. 2023CMMLU: Measuring massive multitask language understanding in Chinesecited by 449
  17. 2024InternLM2 Technical Reportcited by 397
  18. 2023Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Modelscited by 349
  19. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 320
  20. 2024BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domainscited by 301
  21. 2024ReFT: Reasoning with Reinforced Fine-Tuningcited by 239
  22. 2024Marco-o1: Towards Open Reasoning Models for Open-Ended Solutionscited by 144
  23. 2023CMB: A Comprehensive Medical Benchmark in Chinesecited by 115
  24. 2024Forest-of-Thought: Scaling Test-Time Compute for Enhancing LLM Reasoningcited by 110
  25. 2025Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thoughtcited by 109
  26. 2024UltraMedical: Building Specialized Generalists in Biomedicinecited by 67
  27. 2024TreeBoN: Enhancing Inference-Time Alignment with Speculative Tree-Search and Best-of-N Samplingcited by 61
  28. 2024MedExpQA: Multilingual Benchmarking of Large Language Models for Medical Question Answeringcited by 43
  29. 2024System-1.x: Learning to Balance Fast and Slow Planning with Language Modelscited by 35
  30. 2024A Survey on Medical Large Language Models: Technology, Application, Trustworthiness, and Future Directionscited by 34
  31. 2024CodeTree: Agent-guided Tree Search for Code Generation with Large Language Modelscited by 34
  32. 2024AfriMed-QA: A Pan-African, Multi-Specialty, Medical Question-Answering Benchmark Datasetcited by 26
  33. 2024Training on the Benchmark Is Not All You Needcited by 25
  34. 2024MedBench: A Comprehensive, Standardized, and Reliable Benchmarking System for Evaluating Chinese Medical Large Language Modelscited by 21
  35. 2023MedDM:LLM-executable clinical guidance tree for clinical decision-makingcited by 19
  36. 2025Reflection-Window Decoding: Text Generation with Selective Refinementcited by 8
  37. 2024COGNET-MD, an evaluation framework and dataset for Large Language Model benchmarks in the medical domaincited by 4
  38. 2024Towards Training A Chinese Large Language Model for Anesthesiologycited by 2

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026AnesTRACE: Benchmarking Intraoperative Anesthesia from Multimodal Perception to Multi-step Decision-Makingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks
  2. 2024Critique of Impure Reason: Unveiling the reasoning behaviour of medical Large Language Models
  3. 2025Medical Reasoning in the Era of LLMs: A Systematic Review of Enhancement Techniques and Applications
  4. 2025Reasoning LLMs in the Medical Domain: A Literature Survey
  5. 2025Quantifying the Reasoning Abilities of LLMs on Real-world Clinical Cases
  6. 2026Medical Reasoning with Large Language Models: A Survey and MR-Bench
  7. 2025MedHELM: Holistic Evaluation of Large Language Models for Medical Tasks
  8. 2025Automating Expert-Level Medical Reasoning Evaluation of Large Language Models