mlsanityData to 7 Oct 2026

Position: Medical AI Neglects Real Treatment Outcomes

arXiv 2608.14598 ↗First posted 28 Jun 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Medical AI has rapidly improved its ability to perform diagnostic and prognostic tasks that lead to treatment decisions. But understanding of treatment itself is still inadequately trained and evaluated, using human opinions and syntheses (especially texts such as biomedical publications and clinical practice guidelines) rather than actual underlying data on treatment outcomes. This neglect seriously limits the potential of medical AI, and is already causing deficiencies in both frontier models and major benchmarks, as argued in this position paper. Real treatment outcomes, drawn from sources such as observational databases and randomized experiments, should be substantially incorporated into both training and evaluation. Improving these outcomes should be reemphasized as the downstream goal of all medical AI.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 43 of its 137 references that are in the map. First posted on arXiv 28 Jun 2026.

Builds on

43 of 137 references.

References · 43

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  3. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  4. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  5. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  6. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,287
  7. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  8. 2020The Future of Digital Health with Federated Learningcited by 625
  9. 2022MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answeringcited by 525
  10. 2025MedGemma Technical Reportcited by 398
  11. 2025HealthBench: Evaluating Large Language Models Towards Improved Human Healthcited by 385
  12. 2017Snorkel: Rapid Training Data Creation with Weak Supervisioncited by 370
31 more
  1. 2024BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domainscited by 299
  2. 2023MEDITRON-70B: Scaling Medical Pretraining for Large Language Modelscited by 272
  3. 2024Capabilities of Gemini Models in Medicinecited by 231
  4. 2022The 'Problem' of Human Label Variation: On Ground Truth in Data, Modeling and Evaluationcited by 133
  5. 2025HealthGPT: A Medical Large Vision-Language Model for Unifying Comprehension and Generation via Heterogeneous Knowledge Adaptationcited by 112
  6. 2024Benchmarking Large Language Models on Answering and Explaining Challenging Medical Questionscited by 106
  7. 2024Theoretical Foundations of Conformal Predictioncited by 96
  8. 2024Med42-v2: A Suite of Clinical LLMscited by 89
  9. 2021MedPerf: Open Benchmarking Platform for Medical Artificial Intelligence using Federated Evaluationcited by 88
  10. 2021Calibrating Predictions to Decisions: A Novel Approach to Multi-Class Calibrationcited by 79
  11. 2024MEDEC: A Benchmark for Medical Error Detection and Correction in Clinical Notescited by 77
  12. 2023MOTOR: A Time-To-Event Foundation Model For Structured Medical Recordscited by 75
  13. 2024Causal machine learning for predicting treatment outcomescited by 73
  14. 2025TxAgent: An AI Agent for Therapeutic Reasoning Across a Universe of Toolscited by 57
  15. 2021HINT: Hierarchical Interaction Network for Trial Outcome Prediction Leveraging Web Datacited by 49
  16. 2023High-Dimensional Prediction for Sequential Decision Makingcited by 41
  17. 2025Medical Large Language Model Benchmarks Should Prioritize Construct Validitycited by 36
  18. 2022Human Evaluation and Correlation with Automatic Metrics in Consultation Note Generationcited by 33
  19. 2025Generative Medical Event Models Improve with Scalecited by 24
  20. 2026HealthBench Professional: Evaluating Large Language Models on Real Clinician Chatscited by 22
  21. 2025A Modular Approach for Clinical SLMs Driven by Synthetic Data with Pre-Instruction Tuning, Model Merging, and Clinical-Tasks Alignmentcited by 20
  22. 2023Zero-shot causal learningcited by 17
  23. 2022Benchmark datasets driving artificial intelligence development fail to capture the needs of medical professionalscited by 16
  24. 2025Towards Conversational AI for Disease Managementcited by 11
  25. 2026Baichuan-M3: Modeling Clinical Inquiry for Reliable Medical Decision-Makingcited by 10
  26. 2024Automatically Labeling Clinical Trial Outcomes: A Large-Scale Benchmark for Drug Developmentcited by 7
  27. 2022Decision-Making under Miscalibrationcited by 5
  28. 2023Randomization Inference When N Equals Onecited by 5
  29. 2024TrialBench: Multi-Modal Artificial Intelligence-Ready Clinical Trial Datasetscited by 5
  30. 2024Meta-Analysis with Untrusted Datacited by 2
  31. 2025The CRITICAL Records Integrated Standardization Pipeline (CRISP): End-to-End Processing of Large-scale Multi-institutional OMOP CDM Datacited by 1

Built on by

Not cited yet.