mlsanityData to 7 Oct 2026

Let LLMs Take on the Latest Challenges! A Chinese Dynamic Question Answering Benchmark

arXiv 2402.19248 ↗First posted 29 Feb 2024cs.CLcited by 22

Not on the map. Why?

Abstract (arXiv)

How to better evaluate the capabilities of Large Language Models (LLMs) is the focal point and hot topic in current LLMs research. Previous work has noted that due to the extremely high cost of iterative updates of LLMs, they are often unable to answer the latest dynamic questions well. To promote the improvement of Chinese LLMs' ability to answer dynamic questions, in this paper, we introduce CDQA, a Chinese Dynamic QA benchmark containing question-answer pairs related to the latest news on the Chinese Internet. We obtain high-quality data through a pipeline that combines humans and models, and carefully classify the samples according to the frequency of answer changes to facilitate a more fine-grained observation of LLMs' capabilities. We have also evaluated and analyzed mainstream and advanced Chinese LLMs on CDQA. Extensive experiments and valuable insights suggest that our proposed CDQA is challenging and worthy of more further study. We believe that the benchmark we provide will become one of the key data resources for improving LLMs' Chinese question-answering ability in the future.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 24 of its 38 references that are in the map. First posted on arXiv 29 Feb 2024.

Builds on

24 of 38 references.

References · 24

  1. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  2. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  3. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  4. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  5. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  6. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  7. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  8. 2023GAIA: a benchmark for General AI Assistantscited by 1,084
  9. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  10. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 694
  11. 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 663
  12. 2023FreshLLMs: Refreshing Large Language Models with Search Engine Augmentationcited by 308
12 more
  1. 2020CLUE: A Chinese Language Understanding Evaluation Benchmarkcited by 277
  2. 2022RealTime QA: What's the Answer Right Now?cited by 239
  3. 2021A Dataset for Answering Time-Sensitive Questionscited by 150
  4. 2022StreamingQA: A Benchmark for Adaptation to New Knowledge over Time in Question Answering Modelscited by 123
  5. 2023Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselvescited by 123
  6. 2017DuReader: a Chinese Machine Reading Comprehension Dataset from Real-world Applicationscited by 110
  7. 2023WebCPM: Interactive Web Search for Chinese Long-form Question Answeringcited by 93
  8. 2023On the (In)Effectiveness of Large Language Models for Chinese Text Correctioncited by 53
  9. 2023SuperCLUE: A Comprehensive Chinese Large Language Model Benchmarkcited by 50
  10. 2023LatEval: An Interactive LLMs Evaluation Benchmark with Incomplete Information from Lateral Thinking Puzzlescited by 41
  11. 2024When LLMs Meet Cunning Texts: A Fallacy Understanding Benchmark for Large Language Modelscited by 22
  12. 2023SeqGPT: An Out-of-the-box Large Language Model for Open Domain Sequence Understandingcited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 15; 2026: 2.

22 papers.

Sort:

By area:

  1. 2024Evaluation of Retrieval-Augmented Generation: A Surveycited by 146
  2. 2025BrowseComp-ZH: Benchmarking Web Browsing Ability of Large Language Models in Chinesecited by 139
  3. 2024RAGChecker: A Fine-grained Framework for Diagnosing Retrieval-Augmented Generationcited by 59
  4. 2024AntiLeakBench: Preventing Data Contamination by Automatically Constructing Benchmarks with Updated Real-World Knowledgecited by 42
  5. 2025DAST: Context-Aware Compression in LLMs via Dynamic Allocation of Soft Tokenscited by 21
  6. 2025Retrieval Augmented Generation Evaluation in the Era of Large Language Models: A Comprehensive Surveycited by 20
  7. 2025Refine Knowledge of Large Language Models via Adaptive Contrastive Learningcited by 14
  8. 2025One Example Shown, Many Concepts Known! Counterexample-Driven Conceptual Reasoning in Mathematical LLMscited by 12
  9. 2024Query Routing for Homogeneous Tools: An Instantiation in the RAG Scenariocited by 10
  10. 2024ProductAgent: Benchmarking Conversational Product Search Agent with Asking Clarification Questionscited by 10
  11. 2025Memory in Large Language Models: Mechanisms, Evaluation and Evolutioncited by 8
  12. 2025AdmTree: Compressing Lengthy Context with Adaptive Semantic Treescited by 7
10 more
  1. 2026EvoConfig: Self-Evolving Multi-Agent Systems for Efficient Autonomous Environment Configurationcited by 7
  2. 2026Cognitive Mismatch in Multimodal Large Language Models for Discrete Symbol Understandingcited by 6
  3. 2025LexSemBridge: Fine-Grained Dense Representation Enhancement through Token-Aware Embedding Augmentationcited by 5
  4. 2025MDIT: A Model-free Data Interpolation Method for Diverse Instruction Tuningcited by 3
  5. 2025Can we Evaluate RAGs with Synthetic Data?cited by 2
  6. 2025Teaching According to Talents! Instruction Tuning LLMs with Competence-Aware Curriculum Learningcited by 2
  7. 2025Corrections Meet Explanations: A Unified Framework for Explainable Grammatical Error Correctioncited by 1
  8. 2025SEARA: An Automated Approach for Obtaining Optimal Retrieverscited by 0
  9. 2025A Question Answering Dataset for Temporal-Sensitive Retrieval-Augmented Generationcited by 0
  10. 2025CL2GEC: A Multi-Discipline Benchmark for Continual Learning in Chinese Literature Grammatical Error Correctioncited by 0