mlsanityData to 7 Oct 2026

CLEVA: Chinese Language Models EVAluation Platform

arXiv 2308.04813 ↗First posted 9 Aug 2023cs.CLcited by 13

Not on the map. Why?

Abstract (arXiv)

With the continuous emergence of Chinese Large Language Models (LLMs), how to evaluate a model's capabilities has become an increasingly significant issue. The absence of a comprehensive Chinese benchmark that thoroughly assesses a model's performance, the unstandardized and incomparable prompting procedure, and the prevalent risk of contamination pose major challenges in the current evaluation of Chinese LLMs. We present CLEVA, a user-friendly platform crafted to holistically evaluate Chinese LLMs. Our platform employs a standardized workflow to assess LLMs' performance across various dimensions, regularly updating a competitive leaderboard. To alleviate contamination, CLEVA curates a significant proportion of new data and develops a sampling strategy that guarantees a unique subset for each leaderboard round. Empowered by an easy-to-use interface that requires just a few mouse clicks and a model API, users can conduct a thorough evaluation with minimal coding. Large-scale experiments featuring 23 Chinese LLMs have validated CLEVA's efficacy.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 59 of its 95 references that are in the map. First posted on arXiv 9 Aug 2023.

Builds on

59 of 95 references.

References · 59

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  7. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  8. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  9. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  10. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  11. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  12. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
38 more
  1. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  2. 2020Extracting Training Data from Large Language Modelscited by 2,144
  3. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  4. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  5. 2019Language Models as Knowledge Bases?cited by 1,922
  6. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  7. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  8. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  9. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  10. 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,068
  11. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,026
  12. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  13. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 876
  14. 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 701
  15. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 694
  16. 2023CMMLU: Measuring massive multitask language understanding in Chinesecited by 436
  17. 2019Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classificationcited by 396
  18. 2021True Few-Shot Learning with Language Modelscited by 343
  19. 2019Verified Uncertainty Calibrationcited by 307
  20. 2023Efficient and Effective Text Encoding for Chinese LLaMA and Alpacacited by 302
  21. 2022PromptSource: An Integrated Development Environment and Repository for Natural Language Promptscited by 288
  22. 2020CLUE: A Chinese Language Understanding Evaluation Benchmarkcited by 277
  23. 2023PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Promptscited by 218
  24. 2023Inverse Scaling: When Bigger Isn't Bettercited by 205
  25. 2023Evaluating the Performance of Large Language Models on GAOKAO Benchmarkcited by 182
  26. 2022A New Generation of Perspective API: Efficient Multilingual Character-level Transformerscited by 181
  27. 2021OpenPrompt: An Open-source Framework for Prompt-learningcited by 148
  28. 2017ParlAI: A Dialog Research Software Platformcited by 139
  29. 2022A Survey of Deep Learning for Mathematical Reasoningcited by 124
  30. 2018Data-to-Text Generation with Content Selection and Planningcited by 106
  31. 2020OCNLI: Original Chinese Natural Language Inferencecited by 86
  32. 2023Exploring the Impact of Instruction Data Scaling on Large Language Models: An Empirical Study on Real-World Use Casescited by 85
  33. 2023Safety Assessment of Chinese Large Language Modelscited by 82
  34. 2022COLD: A Benchmark for Chinese Offensive Language Detectioncited by 74
  35. 2021NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentationcited by 72
  36. 2020CrossWOZ: A Large-Scale Chinese Cross-Domain Task-Oriented Dialogue Datasetcited by 66
  37. 2019ChID: A Large-scale Chinese IDiom Dataset for Cloze Testcited by 65
  38. 2019Long and Diverse Text Generation with Planning-based Hierarchical Variational Modelcited by 62

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 6; 2025: 2; 2026: 2.

13 papers.

Sort:

By area:

  1. 2023Evaluating Large Language Models: A Comprehensive Surveycited by 200
  2. 2024Large Language Models for Education: A Surveycited by 32
  3. 2024CMMU: A Benchmark for Chinese Multi-modal Multi-type Question Understanding and Reasoningcited by 31
  4. 2023Towards a Psychological Generalist AI: A Survey of Current Applications of Large Language Models and Future Prospectscited by 8
  5. 2023TencentLLMEval: A Hierarchical Evaluation of Real-World Capabilities for Human-Aligned LLMscited by 5
  6. 2024AC-EVAL: Evaluating Ancient Chinese Language Understanding in Large Language Modelscited by 4
  7. 2024Purple-teaming LLMs with Adversarial Defender Trainingcited by 4
  8. 2025F`ux`i: A Benchmark for Evaluating Language Models on Ancient Chinese Text Understanding and Generationcited by 4
  9. 2024C2LEVA: Toward Comprehensive and Contamination-Free Language Model Evaluationcited by 2
  10. 2026TRACER: A Semantic-Aware Framework for Fine-Grained Contamination Detection in Code LLMscited by 1
  11. 2024Code-Based English Models Surprising Performance on Chinese QA Pair Extraction Taskcited by 0
  12. 2025EPT Benchmark: Evaluation of Persian Trustworthiness in Large Language Modelscited by 0
1 more
  1. 2026TaxPraBen: A Scalable Benchmark for Structured Evaluation of LLMs in Chinese Real-World Tax Practicecited by 0