mlsanityData to 14 Oct 2026

MedEthicsQA: A Comprehensive Question Answering Benchmark for Medical Ethics Evaluation of LLMs

arXiv 2506.22808 ↗First posted 28 Jun 2025cs.CL cs.AIcited by 8

Not on the map. Why?

Abstract (arXiv)

While Medical Large Language Models (MedLLMs) have demonstrated remarkable potential in clinical tasks, their ethical safety remains insufficiently explored. This paper introduces MedEthicsQA, a comprehensive benchmark comprising 5,623 multiple-choice questions and 5,351 open-ended questions for evaluation of medical ethics in LLMs. We systematically establish a hierarchical taxonomy integrating global medical ethical standards. The benchmark encompasses widely used medical datasets, authoritative question banks, and scenarios derived from PubMed literature. Rigorous quality control involving multi-stage filtering and multi-faceted expert validation ensures the reliability of the dataset with a low error rate (2.72%). Evaluation of state-of-the-art MedLLMs exhibit declined performance in answering medical ethics questions compared to their foundation counterparts, elucidating the deficiencies of medical ethics alignment. The dataset, registered under CC BY-NC 4.0 license, is available at https://github.com/JianhuiWei7/MedEthicsQA.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 14 of its 38 references that are in the map. First posted on arXiv 28 Jun 2025.

Builds on

14 of 38 references.

References · 14

  1. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  2. 2020Scaling Laws for Neural Language Modelscited by 6,586
  3. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
  4. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,356
  5. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,305
  6. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,288
  7. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 638
  8. 2024HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMscited by 265
  9. 2024Med42-v2: A Suite of Clinical LLMscited by 90
  10. 2024MedCoT: Medical Chain of Thought via Hierarchical Expertcited by 46
  11. 2024LLaVA-Surg: Towards Multimodal Surgical Assistant via Structured Surgical Video Learningcited by 34
  12. 2024JMLR: Joint Medical LLM and Retrieval Training for Enhancing Reasoning and Professional Question Answering Capabilitycited by 32
2 more
  1. 2024Aloe: A Family of Fine-tuned Open Healthcare LLMscited by 20
  2. 2024Guiding Clinical Reasoning with Large Language Models via Knowledge Seedscited by 19

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 6.

8 papers.

  1. 2026From RAG to Agentic RAG for Faithful Islamic Question Answeringcited by 9
  2. 2026PsychEthicsBench: Evaluating Large Language Models Against Australian Mental Health Ethicscited by 4
  3. 2026Assessing Large Language Models for Medical QA: Zero-Shot and LLM-as-a-Judge Evaluationcited by 2
  4. 2026What Does the AI Doctor Value? Auditing Pluralism in the Clinical Ethics of Language Modelscited by 2
  5. 2025Beyond MedQA: Towards Real-world Clinical Decision Making in the Era of LLMscited by 1
  6. 2025PrinciplismQA: A Philosophy-Grounded Approach to Assessing LLM-Human Clinical Medical Ethics Alignmentcited by 0
  7. 2026Agents that Matter: Optimizing Multi-Agent LLMs via Removal-Based Attributioncited by 0
  8. 2026FOCUS: Decoupling Expert Personas in LLMs to Enhance Domain Expert Capabilitiescited by 0