mlsanityData to 14 Oct 2026

Chinese SimpleQA: A Chinese Factuality Evaluation for Large Language Models

arXiv 2411.07140 ↗First posted 11 Nov 2024cs.CLcited by 43

Not on the map. Why?

Abstract (arXiv)

New LLM evaluation benchmarks are important to align with the rapid development of Large Language Models (LLMs). In this work, we present Chinese SimpleQA, the first comprehensive Chinese benchmark to evaluate the factuality ability of language models to answer short questions, and Chinese SimpleQA mainly has five properties (i.e., Chinese, Diverse, High-quality, Static, Easy-to-evaluate). Specifically, first, we focus on the Chinese language over 6 major topics with 99 diverse subtopics. Second, we conduct a comprehensive quality control process to achieve high-quality questions and answers, where the reference answers are static and cannot be changed over time. Third, following SimpleQA, the questions and answers are very short, and the grading process is easy-to-evaluate based on OpenAI API. Based on Chinese SimpleQA, we perform a comprehensive evaluation on the factuality abilities of existing LLMs. Finally, we hope that Chinese SimpleQA could guide the developers to better understand the Chinese factuality abilities of their models and facilitate the growth of foundation models.

Key facts. Cited by 43 papers in the map (data through 14 Oct 2026). First posted on arXiv 11 Nov 2024.

Builds on

0 of 0 references.

No references here yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 26; 2026: 15.

43 papers.

Sort:

By area:

  1. 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 1,040
  2. 2024DeepSeek-V3 Technical Reportcited by 811
  3. 2025Kimi K2: Open Agentic Intelligencecited by 567
  4. 2026MiMo-V2-Flash Technical Reportcited by 191
  5. 2025AdaCoT: Pareto-Optimal Adaptive Chain-of-Thought Triggering via Reinforcement Learningcited by 87
  6. 2025SimpleVQA: Multimodal Factuality Evaluation for Multimodal Large Language Modelscited by 66
  7. 2025When to use Graphs in RAG: A Comprehensive Analysis for Graph Retrieval-Augmented Generationcited by 48
  8. 2025Lost in Benchmarks? Rethinking Large Language Model Benchmarking with Item Response Theorycited by 45
  9. 2025Can Large Language Models Detect Errors in Long Chain-of-Thought Reasoning?cited by 44
  10. 2025CompassVerifier: A Unified and Robust Verifier for LLMs Evaluation and Outcome Rewardcited by 40
  11. 2025Recent Advances in Large Langauge Model Benchmarks against Data Contamination: From Static to Dynamic Evaluationcited by 38
  12. 2025MemSearcher: Training LLMs to Reason, Search and Manage Memory via End-to-End Reinforcement Learningcited by 27
31 more
  1. 2025Every FLOP Counts: Scaling a 300B Mixture-of-Experts LING LLM without Premium GPUscited by 22
  2. 2025Semantic Energy: Detecting LLM Hallucination Beyond Entropycited by 18
  3. 2025Encouraging Good Processes Without the Need for Good Answers: Reinforcement Learning for LLM Agent Planningcited by 17
  4. 2025Co-Sight: Enhancing LLM-Based Agents via Conflict-Aware Meta-Verification and Trustworthy Reasoning with Structured Factscited by 13
  5. 2026ERNIE 5.0 Technical Reportcited by 11
  6. 2024Chinese SafetyQA: A Safety Short-form Factuality Benchmark for Large Language Modelscited by 7
  7. 2026Ling and Ring 2.6 Technical Report: Efficient and Instant Agentic Intelligence at Trillion-Parameter Scalecited by 6
  8. 2025dots.llm1 Technical Reportcited by 5
  9. 2025MAD-Fact: A Multi-Agent Debate Framework for Long-Form Factuality Evaluation in LLMscited by 4
  10. 2026Hidden Decoding at Scale: Latent Computation Scaling for Large Language Modelscited by 4
  11. 2025Unbiased Evaluation of Large Language Models from a Causal Perspectivecited by 3
  12. 2025"See the World, Discover Knowledge": A Chinese Factuality Evaluation for Large Vision Language Modelscited by 3
  13. 2025Evaluating Large Language Model with Knowledge Oriented Language Specific Simple Question Answeringcited by 3
  14. 2025MindVL: Towards Efficient and Effective Training of Multimodal Large Language Models on Ascend NPUscited by 3
  15. 2025Virtual Width Networkscited by 3
  16. 2025Fact or Guesswork? Evaluating Large Language Models' Medical Knowledge with Structured One-Hop Judgmentscited by 2
  17. 2025Beyond Benchmarks: LLM Evaluation with an Anthropomorphic and Lifecycle-oriented Roadmapcited by 1
  18. 2025Benchmarking Chinese Commonsense Reasoning with a Multi-hop Reasoning Perspectivecited by 1
  19. 2025KoSimpleQA: A Korean Factuality Benchmark with an Analysis of Reasoning LLMscited by 1
  20. 2026oHC: Orthogonal Hyper-Connections on SO(4) via Quaternionscited by 1
  21. 2025JT-Safe: Intrinsically Enhancing the Safety and Trustworthiness of LLMscited by 0
  22. 2025LLM Swiss Round: Aggregating Multi-Benchmark Performance via Competitive Swiss-System Dynamicscited by 0
  23. 2026Benchmarking Real-Time Question Answering via Executable Code Workflowscited by 0
  24. 2026Can LLMs Act as Historians? Evaluating Historical Research Capabilities of LLMs via the Chinese Imperial Examinationcited by 0
  25. 2026Continual LLM Upcycling: A Predictor-Gated Bank-Wise Sparsity Training Recipe for Dense-to-Sparse LLMscited by 0
  26. 2026MSQA: A Natively Sourced Multilingual and Multicultural SimpleQA Benchmarkcited by 0
  27. 2026Oyster-II: Reinforcement Learning for Constructive Safety Alignment in Large Language Modelscited by 0
  28. 2026RLVR2: Reinforcement Learning with Verifiable Rubric-based Rankingcited by 0
  29. 2026Trajectory Soup: Pushing the Compute-Scaling Frontier of LLM Mid-training via Diverse Trajectoriescited by 0
  30. 2026Test-time Calibration Learning for Large Language Model Reasoningcited by 0
  31. 2026One Model, Multiple Compute Budgets: Multi-Ratio ConceptMoE for Flexible Compute Across Turnscited by 0