# From Scores to Steps: Diagnosing and Improving LLM Performance in Evidence-Based Medical Calculations

arXiv: https://arxiv.org/abs/2509.16584 · HTML: https://mlsanity.com/paper/2509.16584/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 59 references that are in the map.
- First posted on arXiv 20 Sep 2025.

## Builds on (most cited first, 30 of 30)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [A Survey on LLM-as-a-Judge](https://mlsanity.com/paper/2411.15594/) (2024)
- [What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams](https://mlsanity.com/paper/2009.13081/) (2020)
- [PubMedQA: A Dataset for Biomedical Research Question Answering](https://mlsanity.com/paper/1909.06146/) (2019)
- [Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks](https://mlsanity.com/paper/2211.12588/) (2022)
- [HealthBench: Evaluating Large Language Models Towards Improved Human Health](https://mlsanity.com/paper/2505.08775/) (2025)
- [Large Language Models Are State-of-the-Art Evaluators of Translation Quality](https://mlsanity.com/paper/2302.14520/) (2023)
- [Benchmarking Retrieval-Augmented Generation for Medicine](https://mlsanity.com/paper/2402.13178/) (2024)
- [Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine](https://mlsanity.com/paper/2311.16452/) (2023)
- [MediQ: Question-Asking LLMs and a Benchmark for Reliable Interactive Clinical Reasoning](https://mlsanity.com/paper/2406.00922/) (2024)
- [MedCalc-Bench: Evaluating Large Language Models for Medical Calculations](https://mlsanity.com/paper/2406.12036/) (2024)
- [Evaluating Step-by-step Reasoning Traces: A Survey](https://mlsanity.com/paper/2502.12289/) (2025)
- [Improving Retrieval-Augmented Generation in Medicine with Iterative Follow-up Questions](https://mlsanity.com/paper/2408.00727/) (2024)
- [A Comprehensive Analysis of the Effectiveness of Large Language Models as Automatic Dialogue Evaluators](https://mlsanity.com/paper/2312.15407/) (2023)
- [Summarizing, Simplifying, and Synthesizing Medical Evidence Using GPT-3 (with Varying Success)](https://mlsanity.com/paper/2305.06299/) (2023)
- [JMLR: Joint Medical LLM and Retrieval Training for Enhancing Reasoning and Professional Question Answering Capability](https://mlsanity.com/paper/2402.17887/) (2024)
- [Hidden flaws behind expert-level accuracy of multimodal GPT-4 vision in medicine](https://mlsanity.com/paper/2401.08396/) (2024)
- [From Medprompt to o1: Exploration of Run-Time Strategies for Medical Challenge Problems and Beyond](https://mlsanity.com/paper/2411.03590/) (2024)
- [StoryER: Automatic Story Evaluation via Ranking, Rating and Reasoning](https://mlsanity.com/paper/2210.08459/) (2022)
- [Comparing Two Model Designs for Clinical Note Generation; Is an LLM a Useful Evaluator of Consistency?](https://mlsanity.com/paper/2404.06503/) (2024)
- [Let's Verify Math Questions Step by Step](https://mlsanity.com/paper/2505.13903/) (2025)
- [VeriFact: Verifying Facts in LLM-Generated Clinical Text with Electronic Health Records](https://mlsanity.com/paper/2501.16672/) (2025)
- [MCQG-SRefine: Multiple Choice Question Generation and Evaluation with Iterative Self-Critique, Correction, and Comparison Feedback](https://mlsanity.com/paper/2410.13191/) (2024)
- [Effectiveness of ChatGPT in explaining complex medical reports to patients](https://mlsanity.com/paper/2406.15963/) (2024)

## Built on by (most cited first, 11 of 11)

- [Knowing When to Abstain: Medical LLMs Under Clinical Uncertainty](https://mlsanity.com/paper/2601.12471/) (2026)
- [Scalable Stewardship of an LLM-Assisted Clinical Benchmark with Physician Oversight](https://mlsanity.com/paper/2512.19691/) (2025)
- [The Architecture of Errors: From Universal Impossibility to Patch-Local LLM Reliability](https://mlsanity.com/paper/2605.30628/) (2026)
- [MedCalc-Eval and MedCalc-Env: Advancing Medical Calculation Capabilities of Large Language Models](https://mlsanity.com/paper/2510.27267/) (2025)
- [Flaws in the LLM Automation Narrative](https://mlsanity.com/paper/2606.11166/) (2026)
- [MedBench v5: A Dynamic, Process-Oriented, and Hallucination-Aware Benchmark for Clinical Multimodal Models](https://mlsanity.com/paper/2606.24155/) (2026)
- [MedCalc-Pro: Solving Complex Medical Calculations with LLM Agents](https://mlsanity.com/paper/2607.02879/) (2026)
- [Benchmarking Language Models for Statistical Problem Formulation](https://mlsanity.com/paper/2609.01982/) (2026)
- [Towards a Deterministic Math Solver for Clinical Language Models](https://mlsanity.com/paper/2609.10728/) (2026)
- [Can LLMs Follow Medical Expert Logic? A Benchmark for Hierarchical Logical Consistency in Risk-of-Bias Assessment](https://mlsanity.com/paper/2609.11185/) (2026)
- [Unknown is not normal: separating language-model extraction from rule-based decision logic for clinical risk scores](https://mlsanity.com/paper/2609.34112/) (2026)

## Abstract (arXiv)

Large language models (LLMs) have demonstrated promising performance on medical benchmarks; however, their ability to perform medical calculations, a crucial aspect of clinical decision-making, remains underexplored and poorly evaluated. Existing benchmarks often assess only the final answer with a wide numerical tolerance, overlooking systematic reasoning failures and potentially causing serious clinical misjudgments. In this work, we revisit medical calculation evaluation with a stronger focus on clinical trustworthiness. First, we clean and restructure the MedCalc-Bench dataset and propose a new step-by-step evaluation pipeline that independently assesses formula selection, entity extraction, and arithmetic computation. Under this granular framework, the accuracy of GPT-4o drops from 62.7% to 43.6%, revealing errors masked by prior evaluations. Second, we introduce an automatic error analysis framework that generates structured attribution for each failure mode. Human evaluation confirms its alignment with expert judgment, enabling scalable and explainable diagnostics. Finally, we propose a modular agentic pipeline, MedRaC, that combines retrieval-augmented generation and Python-based code execution. Without any fine-tuning, MedRaC improves the accuracy of different LLMs from 16.35% up to 53.19%. Our work highlights the limitations of current benchmark practices and proposes a more clinically faithful methodology. By enabling transparent and transferable reasoning evaluation, we move closer to making LLM-based systems trustworthy for real-world medical applications.