# MedEthicEval: Evaluating Large Language Models Based on Chinese Medical Ethics

arXiv: https://arxiv.org/abs/2503.02374 · HTML: https://mlsanity.com/paper/2503.02374/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 31 references that are in the map.
- First posted on arXiv 4 Mar 2025.

## Builds on (most cited first, 5 of 5)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Bias and Fairness in Large Language Models: A Survey](https://mlsanity.com/paper/2309.00770/) (2023)
- [Adapted Large Language Models Can Outperform Medical Experts in Clinical Text Summarization](https://mlsanity.com/paper/2309.07430/) (2023)
- [Benchmarking Large Language Models on CMExam -- A Comprehensive Chinese Medical Exam Dataset](https://mlsanity.com/paper/2306.03030/) (2023)
- [Medical Exam Question Answering with Large-scale Reading Comprehension](https://mlsanity.com/paper/1802.10279/) (2018)

## Built on by (most cited first, 8 of 8)

- [From RAG to Agentic RAG for Faithful Islamic Question Answering](https://mlsanity.com/paper/2601.07528/) (2026)
- [EthicsMH: A Pilot Benchmark for Ethical Reasoning in Mental Health AI](https://mlsanity.com/paper/2509.11648/) (2025)
- [Beyond MedQA: Towards Real-world Clinical Decision Making in the Era of LLMs](https://mlsanity.com/paper/2510.20001/) (2025)
- [JMedEthicBench: A Multi-Turn Conversational Benchmark for Evaluating Medical Safety in Japanese Large Language Models](https://mlsanity.com/paper/2601.01627/) (2026)
- [D2VBench: Benchmarking Large Language Models with Value Dilemmas in Daily Scenarios](https://mlsanity.com/paper/2607.19834/) (2026)
- [PrinciplismQA: A Philosophy-Grounded Approach to Assessing LLM-Human Clinical Medical Ethics Alignment](https://mlsanity.com/paper/2508.05132/) (2025)
- [A Human-Centric Pipeline for Aligning Large Language Models with Chinese Medical Ethics](https://mlsanity.com/paper/2601.07954/) (2026)
- [Rare Diseases, Common Dilemmas: LLMs Prioritize Equal Resource Distribution over Patient Benefit in Decision-Making](https://mlsanity.com/paper/2608.25236/) (2026)

## Abstract (arXiv)

Large language models (LLMs) demonstrate significant potential in advancing medical applications, yet their capabilities in addressing medical ethics challenges remain underexplored. This paper introduces MedEthicEval, a novel benchmark designed to systematically evaluate LLMs in the domain of medical ethics. Our framework encompasses two key components: knowledge, assessing the models' grasp of medical ethics principles, and application, focusing on their ability to apply these principles across diverse scenarios. To support this benchmark, we consulted with medical ethics researchers and developed three datasets addressing distinct ethical challenges: blatant violations of medical ethics, priority dilemmas with clear inclinations, and equilibrium dilemmas without obvious resolutions. MedEthicEval serves as a critical tool for understanding LLMs' ethical reasoning in healthcare, paving the way for their responsible and effective use in medical contexts.