mlsanityData to 7 Oct 2026

CLUE: A Chinese Language Understanding Evaluation Benchmark

arXiv 2004.05986 ↗First posted 13 Apr 2020cs.CL cs.LGcited by 277

Not on the map. Why?

Abstract (arXiv)

The advent of natural language understanding (NLU) benchmarks for English, such as GLUE and SuperGLUE allows new NLU models to be evaluated across a diverse set of tasks. These comprehensive benchmarks have facilitated a broad range of research and applications in natural language processing (NLP). The problem, however, is that most such benchmarks are limited to English, which has made it difficult to replicate many of the successes in English NLU for other languages. To help remedy this issue, we introduce the first large-scale Chinese Language Understanding Evaluation (CLUE) benchmark. CLUE is an open-ended, community-driven project that brings together 9 tasks spanning several well-established single-sentence/sentence-pair classification tasks, as well as machine reading comprehension, all on original Chinese text. To establish results on these tasks, we report scores using an exhaustive set of current state-of-the-art pre-trained Chinese models (9 in total). We also introduce a number of supplementary datasets and additional tools to help facilitate further progress on Chinese NLU. Our benchmark is released at https://www.CLUEbenchmarks.com

Key facts. Cited by 277 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 45; 2022: 47; 2023: 66; 2024: 58; 2025: 47; 2026: 14.

277 papers.

Sort:

By area:

  1. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  2. 2023A Survey of Large Language Modelscited by 2,537
  3. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
  4. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,093
  5. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  6. 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 834
  7. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 694
  8. 2023A Comprehensive Overview of Large Language Modelscited by 639
  9. 2024DeepSeek LLM: Scaling Open-Source Language Models with Longtermismcited by 621
  10. 2023C-Pack: Packed Resources For General Chinese Embeddingscited by 556
  11. 2023CMMLU: Measuring massive multitask language understanding in Chinesecited by 436
  12. 2024ShortGPT: Layers in Large Language Models are More Redundant Than You Expectcited by 369
38 more
  1. 2024DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligencecited by 325
  2. 2023PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimizationcited by 311
  3. 2021ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generationcited by 258
  4. 2021PPT: Pre-trained Prompt Tuning for Few-shot Learningcited by 232
  5. 2021Improving Sign Language Translation with Monolingual Data by Sign Back-Translationcited by 177
  6. 2021PanGu-α: Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel Computationcited by 134
  7. 2021AMMUS : A Survey of Transformer-based Pretrained Models in Natural Language Processingcited by 131
  8. 2024LaCo: Large Language Model Pruning via Layer Collapsecited by 124
  9. 2021KLUE: Korean Language Understanding Evaluationcited by 113
  10. 2023LawBench: Benchmarking Legal Knowledge of Large Language Modelscited by 111
  11. 2021CBLUE: A Chinese Biomedical Language Understanding Evaluation Benchmarkcited by 99
  12. 2024Understanding Emergent Abilities of Language Models from the Loss Perspectivecited by 94
  13. 2023XuanYuan 2.0: A Large Chinese Financial Chat Model with Hundreds of Billions Parameterscited by 92
  14. 2021CPT: A Pre-Trained Unbalanced Transformer for Both Chinese Language Understanding and Generationcited by 81
  15. 2021BoB: BERT Over BERT for Training Persona-based Dialogue Models from Limited Personalized Datacited by 75
  16. 2022GLUE-X: Evaluating Natural Language Understanding Models from an Out-of-distribution Generalization Perspectivecited by 72
  17. 2022Fengshenbang 1.0: Being the Foundation of Chinese Cognitive Intelligencecited by 70
  18. 2023LEXTREME: A Multi-Lingual and Multi-Task Benchmark for the Legal Domaincited by 68
  19. 2021ChineseBERT: Chinese Pretraining Enhanced by Glyph and Pinyin Informationcited by 65
  20. 2023Natural Language Reasoning, A Surveycited by 61
  21. 2023BBT-Fin: Comprehensive Construction of Chinese Financial Domain Pre-trained Language Model, Corpus and Benchmarkcited by 60
  22. 2023BigTranslate: Augmenting Large Language Models with Multilingual Translation Capability over 100 Languagescited by 60
  23. 2023PanGu-Sigma: Towards Trillion Parameter Language Model with Sparse Heterogeneous Computingcited by 55
  24. 2024Dynamic Mixture of Experts: An Auto-Tuning Approach for Efficient Transformer Modelscited by 55
  25. 2024Survey on Knowledge Distillation for Large Language Models: Methods, Evaluation, and Applicationcited by 55
  26. 2023Lawyer LLaMA Technical Reportcited by 54
  27. 2025LongCat-Flash Technical Reportcited by 54
  28. 2023FinEval: A Chinese Financial Domain Knowledge Evaluation Benchmark for Large Language Modelscited by 52
  29. 2021HerBERT: Efficiently Pretrained Transformer-based Language Model for Polishcited by 51
  30. 2023SuperCLUE: A Comprehensive Chinese Large Language Model Benchmarkcited by 50
  31. 2022ORCA: A Challenging Benchmark for Arabic Language Understandingcited by 46
  32. 2021Mengzi: Towards Lightweight yet Ingenious Pre-trained Models for Chinesecited by 45
  33. 2021ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generationcited by 45
  34. 2021Towards Efficient NLP: A Standard Evaluation and A Strong Baselinecited by 44
  35. 2021FewCLUE: A Chinese Few-shot Learning Evaluation Benchmarkcited by 41
  36. 2024Should We Respect LLMs? A Cross-Lingual Study on the Influence of Prompt Politeness on LLM Performancecited by 41
  37. 2023Benchmarks for Automated Commonsense Reasoning: A Surveycited by 40
  38. 2023Origin Tracing and Detecting of LLMscited by 39

50 most cited of 277 All 277 (JSON)