# CMoralEval: A Moral Evaluation Benchmark for Chinese Large Language Models

arXiv: https://arxiv.org/abs/2408.09819 · HTML: https://mlsanity.com/paper/2408.09819/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 42 references that are in the map.
- First posted on arXiv 19 Aug 2024.

## Builds on (most cited first, 20 of 20)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [GLM: General Language Model Pretraining with Autoregressive Blank Infilling](https://mlsanity.com/paper/2103.10360/) (2021)
- [GLM-130B: An Open Bilingual Pre-trained Model](https://mlsanity.com/paper/2210.02414/) (2022)
- [Aligning AI With Shared Human Values](https://mlsanity.com/paper/2008.02275/) (2020)
- [BBQ: A Hand-Built Bias Benchmark for Question Answering](https://mlsanity.com/paper/2110.08193/) (2021)
- [Baichuan 2: Open Large-scale Language Models](https://mlsanity.com/paper/2309.10305/) (2023)
- [Efficient and Effective Text Encoding for Chinese LLaMA and Alpaca](https://mlsanity.com/paper/2304.08177/) (2023)
- [Social Chemistry 101: Learning to Reason about Social and Moral Norms](https://mlsanity.com/paper/2011.00620/) (2020)
- [Large Language Model Alignment: A Survey](https://mlsanity.com/paper/2309.15025/) (2023)
- [Evaluating the Moral Beliefs Encoded in LLMs](https://mlsanity.com/paper/2307.14324/) (2023)
- [Evaluating Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2310.19736/) (2023)
- [On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning](https://mlsanity.com/paper/2212.08061/) (2022)
- [Moral Stories: Situated Reasoning about Norms, Intents, Actions, and their Consequences](https://mlsanity.com/paper/2012.15738/) (2020)
- [ProsocialDialog: A Prosocial Backbone for Conversational Agents](https://mlsanity.com/paper/2205.12688/) (2022)
- [Scruples: A Corpus of Community Ethical Judgments on 32,000 Real-Life Anecdotes](https://mlsanity.com/paper/2008.09094/) (2020)
- [The Moral Integrity Corpus: A Benchmark for Ethical Dialogue Systems](https://mlsanity.com/paper/2204.03021/) (2022)
- [When to Make Exceptions: Exploring Language Models as Accounts of Human Moral Judgment](https://mlsanity.com/paper/2210.01478/) (2022)
- [LMFlow: An Extensible Toolkit for Finetuning and Inference of Large Foundation Models](https://mlsanity.com/paper/2306.12420/) (2023)
- [FFB: A Fair Fairness Benchmark for In-Processing Group Fairness Methods](https://mlsanity.com/paper/2306.09468/) (2023)
- [TigerBot: An Open Multilingual Multitask LLM](https://mlsanity.com/paper/2312.08688/) (2023)

## Built on by (most cited first, 10 of 10)

- [Accumulating Context Changes the Beliefs of Language Models](https://mlsanity.com/paper/2511.01805/) (2025)
- [The Staircase of Ethics: Probing LLM Value Priorities through Multi-Step Induction to Complex Moral Dilemmas](https://mlsanity.com/paper/2505.18154/) (2025)
- [MM-MoralBench: A MultiModal Moral Evaluation Benchmark for Large Vision-Language Models](https://mlsanity.com/paper/2412.20718/) (2024)
- [Isolating Culture Neurons in Multilingual Large Language Models](https://mlsanity.com/paper/2508.02241/) (2025)
- [One Model, Many Morals: Uncovering Cross-Linguistic Misalignments in Computational Moral Reasoning](https://mlsanity.com/paper/2509.21443/) (2025)
- [C-VARC: A Large-Scale Chinese Value Rule Corpus for Value Alignment of Large Language Models](https://mlsanity.com/paper/2506.01495/) (2025)
- [Do Large Language Models Know Folktales? A Case Study of Yokai in Japanese Folktales](https://mlsanity.com/paper/2506.03619/) (2025)
- [The yes-no bias of large language models reflects answer order and wording, not shifts in moral judgment](https://mlsanity.com/paper/2607.05552/) (2026)
- [D2VBench: Benchmarking Large Language Models with Value Dilemmas in Daily Scenarios](https://mlsanity.com/paper/2607.19834/) (2026)
- [Retrieval-Augmented Large Language Model Decision-Making for Autonomous Driving Guided by Chinese Philosophical Wisdom](https://mlsanity.com/paper/2610.03948/) (2026)

## Abstract (arXiv)

What a large language model (LLM) would respond in ethically relevant context? In this paper, we curate a large benchmark CMoralEval for morality evaluation of Chinese LLMs. The data sources of CMoralEval are two-fold: 1) a Chinese TV program discussing Chinese moral norms with stories from the society and 2) a collection of Chinese moral anomies from various newspapers and academic papers on morality. With these sources, we aim to create a moral evaluation dataset characterized by diversity and authenticity. We develop a morality taxonomy and a set of fundamental moral principles that are not only rooted in traditional Chinese culture but also consistent with contemporary societal norms. To facilitate efficient construction and annotation of instances in CMoralEval, we establish a platform with AI-assisted instance generation to streamline the annotation process. These help us curate CMoralEval that encompasses both explicit moral scenarios (14,964 instances) and moral dilemma scenarios (15,424 instances), each with instances from different data sources. We conduct extensive experiments with CMoralEval to examine a variety of Chinese LLMs. Experiment results demonstrate that CMoralEval is a challenging benchmark for Chinese LLMs. The dataset is publicly available at urlhttps://github.com/tjunlp-lab/CMoralEval.