mlsanityData to 7 Oct 2026

LiveClin: A Live Clinical Benchmark without Leakage

arXiv 2602.16747 ↗First posted 18 Feb 2026cs.LG cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

The reliability of medical LLM evaluation is critically undermined by data contamination and knowledge obsolescence, leading to inflated scores on static benchmarks. To address these challenges, we introduce LiveClin, a live benchmark designed for approximating real-world clinical practice. Built from contemporary, peer-reviewed case reports and updated biannually, LiveClin ensures clinical currency and resists data contamination. Using a verified AI-human workflow involving 239 physicians, we transform authentic patient cases into complex, multimodal evaluation scenarios that span the entire clinical pathway. The benchmark currently comprises 1,407 case reports and 6,605 questions. Our evaluation of 26 models on LiveClin reveals the profound difficulty of these real-world scenarios, with the top-performing model achieving a Case Accuracy of just 35.7%. In benchmarking against human experts, Chief Physicians achieved the highest accuracy, followed closely by Attending Physicians, with both surpassing most models. LiveClin thus provides a continuously evolving, clinically grounded framework to guide the development of medical LLMs towards closing this gap and achieving greater reliability and real-world utility. Our data and code are publicly available at https://github.com/AQ-MedAI/LiveClin.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 13 of its 25 references that are in the map. First posted on arXiv 18 Feb 2026.

Builds on

13 of 25 references.

References · 13

  1. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  2. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
  3. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  4. 2020PathVQA: 30000+ Questions for Medical Visual Question Answeringcited by 467
  5. 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learningcited by 439
  6. 2025MathArena: Evaluating LLMs on Uncontaminated Math Competitionscited by 275
  7. 2024LiveBench: A Challenging, Contamination-Limited LLM Benchmarkcited by 242
  8. 2023Proving Test Set Contamination in Black Box Language Modelscited by 227
  9. 2024AgentClinic: a multimodal agent benchmark to evaluate AI in simulated clinical environmentscited by 215
  10. 2023Investigating Data Contamination in Modern Benchmarks for Large Language Modelscited by 136
  11. 2024ReXrank: A Public Leaderboard for AI-Powered Radiology Report Generationcited by 24
  12. 2024Inference-Time Decontamination: Reusing Leaked Benchmarks for Large Language Model Evaluationcited by 18
1 more
  1. 2025CaseReportBench: An LLM Benchmark Dataset for Dense Information Extraction in Clinical Case Reportscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.