mlsanityData to 7 Oct 2026

CValues: Measuring the Values of Chinese Large Language Models from Safety to Responsibility

arXiv 2307.09705 ↗First posted 19 Jul 2023cs.CLcited by 80

Not on the map. Why?

Abstract (arXiv)

With the rapid evolution of large language models (LLMs), there is a growing concern that they may pose risks or have negative social impacts. Therefore, evaluation of human values alignment is becoming increasingly important. Previous work mainly focuses on assessing the performance of LLMs on certain knowledge and reasoning abilities, while neglecting the alignment to human values, especially in a Chinese context. In this paper, we present CValues, the first Chinese human values evaluation benchmark to measure the alignment ability of LLMs in terms of both safety and responsibility criteria. As a result, we have manually collected adversarial safety prompts across 10 scenarios and induced responsibility prompts from 8 domains by professional experts. To provide a comprehensive values evaluation of Chinese LLMs, we not only conduct human evaluation for reliable comparison, but also construct multi-choice prompts for automatic evaluation. Our findings suggest that while most Chinese LLMs perform well in terms of safety, there is considerable room for improvement in terms of responsibility. Moreover, both the automatic and human evaluation are important for assessing the human values alignment in different aspects. The benchmark and code is available on ModelScope and Github.

Key facts. Cited by 80 papers in the map (data through 7 Oct 2026). Builds on 25 of its 36 references that are in the map. First posted on arXiv 19 Jul 2023.

Builds on

25 of 36 references.

References · 25

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  7. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  8. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  9. 2017Deep reinforcement learning from human preferencescited by 3,846
  10. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  11. 2015Teaching Machines to Read and Comprehendcited by 1,039
  12. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
13 more
  1. 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 701
  2. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 694
  3. 2023PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimizationcited by 311
  4. 2023Efficient and Effective Text Encoding for Chinese LLaMA and Alpacacited by 302
  5. 2021ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generationcited by 258
  6. 2023Evaluating the Performance of Large Language Models on GAOKAO Benchmarkcited by 182
  7. 2021PanGu-α: Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel Computationcited by 134
  8. 2023Safety Assessment of Chinese Large Language Modelscited by 82
  9. 2021On the Safety of Conversational Models: Taxonomy, Dataset, and Benchmarkcited by 73
  10. 2022Fengshenbang 1.0: Being the Foundation of Chinese Cognitive Intelligencecited by 70
  11. 2023M3KE: A Massive Multi-Level Multi-Subject Knowledge Evaluation Benchmark for Chinese Large Language Modelscited by 30
  12. 2023Measuring Massive Multitask Chinese Understandingcited by 25
  13. 2023ChatPLUG: Open-Domain Generative Dialogue System with Internet-Augmented Instruction Tuning for Digital Humancited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 14; 2024: 38; 2025: 22; 2026: 6.

80 papers.

Sort:

By area:

  1. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 520
  2. 2024LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methodscited by 451
  3. 2023Benchmarking Large Language Models in Retrieval-Augmented Generationcited by 307
  4. 2024Agent-SafetyBench: Evaluating the Safety of LLM Agentscited by 280
  5. 2024SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Modelscited by 225
  6. 2023SafetyBench: Evaluating the Safety of Large Language Modelscited by 189
  7. 2023Can LLM-Generated Misinformation Be Detected?cited by 173
  8. 2024Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Modelscited by 162
  9. 2025LLM Post-Training: A Deep Dive into Reasoning Large Language Modelscited by 129
  10. 2024Know Your Limits: A Survey of Abstention in Large Language Modelscited by 116
  11. 2024SafetyPrompts: a Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safetycited by 75
  12. 2024AIR-Bench 2024: A Safety Benchmark Based on Risk Categories from Regulations and Policiescited by 69
38 more
  1. 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systemscited by 63
  2. 2025Safety at Scale: A Comprehensive Survey of Large Model and Agent Safetycited by 63
  3. 2023Value FULCRA: Mapping Large Language Models to the Multidimensional Spectrum of Basic Human Valuescited by 49
  4. 2024CultureLLM: Incorporating Cultural Differences into Large Language Modelscited by 49
  5. 2024ShieldLM: Empowering LLMs as Aligned, Customizable and Explainable Safety Detectorscited by 48
  6. 2024ALI-Agent: Assessing LLMs' Alignment with Human Values via Agent-based Evaluationcited by 46
  7. 2023From Instructions to Intrinsic Human Values -- A Survey of Alignment Goals for Big Modelscited by 45
  8. 2024COIG-CQIA: Quality is All You Need for Chinese Instruction Fine-tuningcited by 45
  9. 2023ChiMed-GPT: A Chinese Medical Large Language Model with Full Training Regime and Better Alignment to Human Preferencescited by 43
  10. 2023Fake Alignment: Are LLMs Really Aligned Well?cited by 39
  11. 2023Flames: Benchmarking Value Alignment of LLMs in Chinesecited by 35
  12. 2024ROSE Doesn't Do That: Boosting the Safety of Instruction-Tuned Large Language Models with Reverse Prompt Contrastive Decodingcited by 32
  13. 2024Can Large Language Models Understand Symbolic Graphics Programs?cited by 32
  14. 2024Audio Is the Achilles' Heel: Red Teaming Audio Large Multimodal Modelscited by 31
  15. 2024S-Eval: Towards Automated and Comprehensive Safety Evaluation for Large Language Modelscited by 28
  16. 2024A Survey of Useful LLM Evaluationcited by 28
  17. 2024Reuse Your Rewards: Reward Model Transfer for Zero-Shot Cross-Lingual Alignmentcited by 26
  18. 2024CulturePark: Boosting Cross-cultural Understanding in Large Language Modelscited by 24
  19. 2025The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMscited by 24
  20. 2023Goal-Oriented Prompt Attack and Safety Evaluation for LLMscited by 23
  21. 2023CDEval: A Benchmark for Measuring the Cultural Dimensions of Large Language Modelscited by 23
  22. 2024CLAVE: An Adaptive Framework for Evaluating Values of LLM Generated Responsescited by 23
  23. 2024From LLMs to MLLMs: Exploring the Landscape of Multimodal Jailbreakingcited by 22
  24. 2025Every FLOP Counts: Scaling a 300B Mixture-of-Experts LING LLM without Premium GPUscited by 22
  25. 2024Causal Evaluation of Language Modelscited by 18
  26. 2023SC-Safety: A Multi-round Open-ended Question Adversarial Safety Benchmark for Large Language Models in Chinesecited by 17
  27. 2025A Survey of Foundation Model-Powered Recommender Systems: From Feature-Based, Generative to Agentic Paradigmscited by 17
  28. 2024AraTrust: An Evaluation of Trustworthiness for LLMs in Arabiccited by 16
  29. 2024CIVICS: Building a Dataset for Examining Culturally-Informed Values in Large Language Modelscited by 16
  30. 2024Jigsaw Puzzles: Splitting Harmful Questions to Jailbreak Large Language Modelscited by 15
  31. 2024CHiSafetyBench: A Chinese Hierarchical Safety Benchmark for Large Language Modelscited by 13
  32. 2024Dialectical Alignment: Resolving the Tension of 3H and Security Threats of LLMscited by 11
  33. 2025Building Safe GenAI Applications: An End-to-End Overview of Red Teaming for Large Language Modelscited by 11
  34. 2023The Good, The Bad, and Why: Unveiling Emotions in Generative AIcited by 10
  35. 2025From Surveys to Narratives: Rethinking Cultural Value Adaptation in LLMscited by 9
  36. 2026Agentic Abstention: Do Agents Know When to Stop Instead of Act?cited by 9
  37. 2024NewsBench: A Systematic Evaluation Framework for Assessing Editorial Capabilities of Large Language Models in Chinese Journalismcited by 8
  38. 2025CLINIC: Evaluating Multilingual Trustworthiness in Language Models for Healthcarecited by 7

50 most cited of 80 All 80 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023TencentLLMEval: A Hierarchical Evaluation of Real-World Capabilities for Human-Aligned LLMs
  2. 2026Adversarial Alignment: Ensuring Value Consistency in Large Language Models for Sensitive Domains
  3. 2025Cultural Value Alignment in Large Language Models: A Prompt-based Analysis of Schwartz Values in Gemini, ChatGPT, and DeepSeek
  4. 2025Diverse Human Value Alignment for Large Language Models via Ethical Reasoning
  5. 2025DiverValue-Bench: A Benchmark and Fine-Tuning Framework for Aligning Large Language Models with Diverse Human Values
  6. 2024OpenEval: Benchmarking Chinese LLMs across Capability, Alignment and Safety
  7. 2024LocalValueBench: A Collaboratively Built and Extensible Benchmark for Evaluating Localized Value Alignment and Ethical Safety in Large Language Models
  8. 2026Beyond Value Benchmarks: Measuring Value-Structure Alignment in Large Language Models via Symmetric Q-Sorts