# CValues: Measuring the Values of Chinese Large Language Models from Safety to Responsibility

arXiv: https://arxiv.org/abs/2307.09705 · HTML: https://mlsanity.com/paper/2307.09705/

## Key facts

- Cited by 85 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 36 references that are in the map.
- First posted on arXiv 19 Jul 2023.

## Builds on (most cited first, 25 of 25)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [Teaching Machines to Read and Comprehend](https://mlsanity.com/paper/1506.03340/) (2015)
- [GLM-130B: An Open Bilingual Pre-trained Model](https://mlsanity.com/paper/2210.02414/) (2022)
- [AGIEval: A Human-Centric Benchmark for Evaluating Foundation Models](https://mlsanity.com/paper/2304.06364/) (2023)
- [C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Models](https://mlsanity.com/paper/2305.08322/) (2023)
- [PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimization](https://mlsanity.com/paper/2306.05087/) (2023)
- [Efficient and Effective Text Encoding for Chinese LLaMA and Alpaca](https://mlsanity.com/paper/2304.08177/) (2023)
- [ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation](https://mlsanity.com/paper/2107.02137/) (2021)
- [Evaluating the Performance of Large Language Models on GAOKAO Benchmark](https://mlsanity.com/paper/2305.12474/) (2023)
- [PanGu-α: Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel Computation](https://mlsanity.com/paper/2104.12369/) (2021)
- [Safety Assessment of Chinese Large Language Models](https://mlsanity.com/paper/2304.10436/) (2023)
- [On the Safety of Conversational Models: Taxonomy, Dataset, and Benchmark](https://mlsanity.com/paper/2110.08466/) (2021)
- [Fengshenbang 1.0: Being the Foundation of Chinese Cognitive Intelligence](https://mlsanity.com/paper/2209.02970/) (2022)
- [M3KE: A Massive Multi-Level Multi-Subject Knowledge Evaluation Benchmark for Chinese Large Language Models](https://mlsanity.com/paper/2305.10263/) (2023)
- [Measuring Massive Multitask Chinese Understanding](https://mlsanity.com/paper/2304.12986/) (2023)
- [ChatPLUG: Open-Domain Generative Dialogue System with Internet-Augmented Instruction Tuning for Digital Human](https://mlsanity.com/paper/2304.07849/) (2023)

## Built on by (most cited first, 30 of 85)

- [GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts](https://mlsanity.com/paper/2309.10253/) (2023)
- [LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods](https://mlsanity.com/paper/2412.05579/) (2024)
- [Benchmarking Large Language Models in Retrieval-Augmented Generation](https://mlsanity.com/paper/2309.01431/) (2023)
- [Agent-SafetyBench: Evaluating the Safety of LLM Agents](https://mlsanity.com/paper/2412.14470/) (2024)
- [SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models](https://mlsanity.com/paper/2402.05044/) (2024)
- [SafetyBench: Evaluating the Safety of Large Language Models](https://mlsanity.com/paper/2309.07045/) (2023)
- [Can LLM-Generated Misinformation Be Detected?](https://mlsanity.com/paper/2309.13788/) (2023)
- [Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models](https://mlsanity.com/paper/2402.16786/) (2024)
- [LLM Post-Training: A Deep Dive into Reasoning Large Language Models](https://mlsanity.com/paper/2502.21321/) (2025)
- [Know Your Limits: A Survey of Abstention in Large Language Models](https://mlsanity.com/paper/2407.18418/) (2024)
- [SafetyPrompts: a Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safety](https://mlsanity.com/paper/2404.05399/) (2024)
- [AIR-Bench 2024: A Safety Benchmark Based on Risk Categories from Regulations and Policies](https://mlsanity.com/paper/2407.17436/) (2024)
- [Safety at Scale: A Comprehensive Survey of Large Model and Agent Safety](https://mlsanity.com/paper/2502.05206/) (2025)
- [Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems](https://mlsanity.com/paper/2401.05778/) (2024)
- [Large Language Model Safety: A Holistic Survey](https://mlsanity.com/paper/2412.17686/) (2024)
- [ShieldLM: Empowering LLMs as Aligned, Customizable and Explainable Safety Detectors](https://mlsanity.com/paper/2402.16444/) (2024)
- [Value FULCRA: Mapping Large Language Models to the Multidimensional Spectrum of Basic Human Values](https://mlsanity.com/paper/2311.10766/) (2023)
- [CultureLLM: Incorporating Cultural Differences into Large Language Models](https://mlsanity.com/paper/2402.10946/) (2024)
- [ALI-Agent: Assessing LLMs' Alignment with Human Values via Agent-based Evaluation](https://mlsanity.com/paper/2405.14125/) (2024)
- [From Instructions to Intrinsic Human Values -- A Survey of Alignment Goals for Big Models](https://mlsanity.com/paper/2308.12014/) (2023)
- [COIG-CQIA: Quality is All You Need for Chinese Instruction Fine-tuning](https://mlsanity.com/paper/2403.18058/) (2024)
- [ChiMed-GPT: A Chinese Medical Large Language Model with Full Training Regime and Better Alignment to Human Preferences](https://mlsanity.com/paper/2311.06025/) (2023)
- [Fake Alignment: Are LLMs Really Aligned Well?](https://mlsanity.com/paper/2311.05915/) (2023)
- [Flames: Benchmarking Value Alignment of LLMs in Chinese](https://mlsanity.com/paper/2311.06899/) (2023)
- [Audio Is the Achilles' Heel: Red Teaming Audio Large Multimodal Models](https://mlsanity.com/paper/2410.23861/) (2024)
- [Can Large Language Models Understand Symbolic Graphics Programs?](https://mlsanity.com/paper/2408.08313/) (2024)
- [S-Eval: Towards Automated and Comprehensive Safety Evaluation for Large Language Models](https://mlsanity.com/paper/2405.14191/) (2024)
- [ROSE Doesn't Do That: Boosting the Safety of Instruction-Tuned Large Language Models with Reverse Prompt Contrastive Decoding](https://mlsanity.com/paper/2402.11889/) (2024)
- [A Survey of Useful LLM Evaluation](https://mlsanity.com/paper/2406.00936/) (2024)
- [Reuse Your Rewards: Reward Model Transfer for Zero-Shot Cross-Lingual Alignment](https://mlsanity.com/paper/2404.12318/) (2024)

## Abstract (arXiv)

With the rapid evolution of large language models (LLMs), there is a growing concern that they may pose risks or have negative social impacts. Therefore, evaluation of human values alignment is becoming increasingly important. Previous work mainly focuses on assessing the performance of LLMs on certain knowledge and reasoning abilities, while neglecting the alignment to human values, especially in a Chinese context. In this paper, we present CValues, the first Chinese human values evaluation benchmark to measure the alignment ability of LLMs in terms of both safety and responsibility criteria. As a result, we have manually collected adversarial safety prompts across 10 scenarios and induced responsibility prompts from 8 domains by professional experts. To provide a comprehensive values evaluation of Chinese LLMs, we not only conduct human evaluation for reliable comparison, but also construct multi-choice prompts for automatic evaluation. Our findings suggest that while most Chinese LLMs perform well in terms of safety, there is considerable room for improvement in terms of responsibility. Moreover, both the automatic and human evaluation are important for assessing the human values alignment in different aspects. The benchmark and code is available on ModelScope and Github.