A Course Shared Task on Evaluating LLM Output for Clinical Questions
Not on the map. Why?
Abstract (arXiv)
This paper presents a shared task that we organized at the Foundations of Language Technology (FoLT) course in 2023/2024 at the Technical University of Darmstadt, which focuses on evaluating the output of Large Language Models (LLMs) in generating harmful answers to health-related clinical questions. We describe the task design considerations and report the feedback we received from the students. We expect the task and the findings reported in this paper to be relevant for instructors teaching natural language processing (NLP) and designing course assignments.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 7 of its 9 references that are in the map. First posted on arXiv 31 Jul 2024.
Builds on
7 of 9 references.
References · 7
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
- 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
- 2020Fact or Fiction: Verifying Scientific Claimscited by 546
- 2020Explainable Automated Fact-Checking for Public Health Claimscited by 209
- 2022Missing Counter-Evidence Renders NLP Fact-Checking Unrealistic for Misinformationcited by 45
- 2024WikiContradict: A Benchmark for Evaluating LLMs on Real-World Knowledge Conflicts from Wikipediacited by 43
- 2024Missci: Reconstructing Fallacies in Misrepresented Sciencecited by 14
Built on by
Not cited yet.