mlsanityData to 14 Oct 2026

Overview of the ClinIQLink 2025 Shared Task on Medical Question-Answering

arXiv 2506.21597 ↗First posted 18 Jun 2025cs.CL cs.AI cs.IRcited by 3

Not on the map. Why?

Abstract (arXiv)

In this paper, we present an overview of ClinIQLink, a shared task, collocated with the 24th BioNLP workshop at ACL 2025, designed to stress-test large language models (LLMs) on medically-oriented question answering aimed at the level of a General Practitioner. The challenge supplies 4,978 expert-verified, medical source-grounded question-answer pairs that cover seven formats: true/false, multiple choice, unordered list, short answer, short-inverse, multi-hop, and multi-hop-inverse. Participating systems, bundled in Docker or Apptainer images, are executed on the CodaBench platform or the University of Maryland's Zaratan cluster. An automated harness (Task 1) scores closed-ended items by exact match and open-ended items with a three-tier embedding metric. A subsequent physician panel (Task 2) audits the top model responses.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 3 of its 15 references that are in the map. First posted on arXiv 18 Jun 2025.

Builds on

3 of 15 references.

References · 3

  1. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,178
  2. 2023The Falcon Series of Open Language Modelscited by 535
  3. 2025Reference-free Evaluation Metrics for Text Generation: A Surveycited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Overview of the PsyDefDetect Shared Task at BioNLP 2026: Detecting Levels of Psychological Defense Mechanisms in Supportive Conversationscited by 3
  2. 2026MedHopQA: A Disease-Centered Multi-Hop Reasoning Benchmark and Evaluation Framework for LLM-Based Biomedical Question Answeringcited by 2
  3. 2026ERRORQUAKE: Heavy-Tailed Error Severity Distributions in Open-Weight Large Language Modelscited by 0