# Let LLMs Take on the Latest Challenges! A Chinese Dynamic Question Answering Benchmark

arXiv: https://arxiv.org/abs/2402.19248 · HTML: https://mlsanity.com/paper/2402.19248/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 38 references that are in the map.
- First posted on arXiv 29 Feb 2024.

## Builds on (most cited first, 24 of 24)

- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge](https://mlsanity.com/paper/1811.00937/) (2018)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [GAIA: a benchmark for General AI Assistants](https://mlsanity.com/paper/2311.12983/) (2023)
- [Reading Wikipedia to Answer Open-Domain Questions](https://mlsanity.com/paper/1704.00051/) (2017)
- [C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Models](https://mlsanity.com/paper/2305.08322/) (2023)
- [Unifying Large Language Models and Knowledge Graphs: A Roadmap](https://mlsanity.com/paper/2306.08302/) (2023)
- [FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation](https://mlsanity.com/paper/2310.03214/) (2023)
- [CLUE: A Chinese Language Understanding Evaluation Benchmark](https://mlsanity.com/paper/2004.05986/) (2020)
- [RealTime QA: What's the Answer Right Now?](https://mlsanity.com/paper/2207.13332/) (2022)
- [A Dataset for Answering Time-Sensitive Questions](https://mlsanity.com/paper/2108.06314/) (2021)
- [StreamingQA: A Benchmark for Adaptation to New Knowledge over Time in Question Answering Models](https://mlsanity.com/paper/2205.11388/) (2022)
- [Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselves](https://mlsanity.com/paper/2311.04205/) (2023)
- [DuReader: a Chinese Machine Reading Comprehension Dataset from Real-world Applications](https://mlsanity.com/paper/1711.05073/) (2017)
- [WebCPM: Interactive Web Search for Chinese Long-form Question Answering](https://mlsanity.com/paper/2305.06849/) (2023)
- [On the (In)Effectiveness of Large Language Models for Chinese Text Correction](https://mlsanity.com/paper/2307.09007/) (2023)
- [SuperCLUE: A Comprehensive Chinese Large Language Model Benchmark](https://mlsanity.com/paper/2307.15020/) (2023)
- [LatEval: An Interactive LLMs Evaluation Benchmark with Incomplete Information from Lateral Thinking Puzzles](https://mlsanity.com/paper/2308.10855/) (2023)
- [When LLMs Meet Cunning Texts: A Fallacy Understanding Benchmark for Large Language Models](https://mlsanity.com/paper/2402.11100/) (2024)
- [SeqGPT: An Out-of-the-box Large Language Model for Open Domain Sequence Understanding](https://mlsanity.com/paper/2308.10529/) (2023)

## Built on by (most cited first, 22 of 22)

- [Evaluation of Retrieval-Augmented Generation: A Survey](https://mlsanity.com/paper/2405.07437/) (2024)
- [BrowseComp-ZH: Benchmarking Web Browsing Ability of Large Language Models in Chinese](https://mlsanity.com/paper/2504.19314/) (2025)
- [RAGChecker: A Fine-grained Framework for Diagnosing Retrieval-Augmented Generation](https://mlsanity.com/paper/2408.08067/) (2024)
- [AntiLeakBench: Preventing Data Contamination by Automatically Constructing Benchmarks with Updated Real-World Knowledge](https://mlsanity.com/paper/2412.13670/) (2024)
- [DAST: Context-Aware Compression in LLMs via Dynamic Allocation of Soft Tokens](https://mlsanity.com/paper/2502.11493/) (2025)
- [Retrieval Augmented Generation Evaluation in the Era of Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2504.14891/) (2025)
- [Refine Knowledge of Large Language Models via Adaptive Contrastive Learning](https://mlsanity.com/paper/2502.07184/) (2025)
- [One Example Shown, Many Concepts Known! Counterexample-Driven Conceptual Reasoning in Mathematical LLMs](https://mlsanity.com/paper/2502.10454/) (2025)
- [Query Routing for Homogeneous Tools: An Instantiation in the RAG Scenario](https://mlsanity.com/paper/2406.12429/) (2024)
- [ProductAgent: Benchmarking Conversational Product Search Agent with Asking Clarification Questions](https://mlsanity.com/paper/2407.00942/) (2024)
- [Memory in Large Language Models: Mechanisms, Evaluation and Evolution](https://mlsanity.com/paper/2509.18868/) (2025)
- [AdmTree: Compressing Lengthy Context with Adaptive Semantic Trees](https://mlsanity.com/paper/2512.04550/) (2025)
- [EvoConfig: Self-Evolving Multi-Agent Systems for Efficient Autonomous Environment Configuration](https://mlsanity.com/paper/2601.16489/) (2026)
- [Cognitive Mismatch in Multimodal Large Language Models for Discrete Symbol Understanding](https://mlsanity.com/paper/2603.18472/) (2026)
- [LexSemBridge: Fine-Grained Dense Representation Enhancement through Token-Aware Embedding Augmentation](https://mlsanity.com/paper/2508.17858/) (2025)
- [MDIT: A Model-free Data Interpolation Method for Diverse Instruction Tuning](https://mlsanity.com/paper/2504.07288/) (2025)
- [Can we Evaluate RAGs with Synthetic Data?](https://mlsanity.com/paper/2508.11758/) (2025)
- [Teaching According to Talents! Instruction Tuning LLMs with Competence-Aware Curriculum Learning](https://mlsanity.com/paper/2509.13790/) (2025)
- [Corrections Meet Explanations: A Unified Framework for Explainable Grammatical Error Correction](https://mlsanity.com/paper/2502.15261/) (2025)
- [SEARA: An Automated Approach for Obtaining Optimal Retrievers](https://mlsanity.com/paper/2507.06554/) (2025)
- [A Question Answering Dataset for Temporal-Sensitive Retrieval-Augmented Generation](https://mlsanity.com/paper/2508.12282/) (2025)
- [CL2GEC: A Multi-Discipline Benchmark for Continual Learning in Chinese Literature Grammatical Error Correction](https://mlsanity.com/paper/2509.13672/) (2025)

## Abstract (arXiv)

How to better evaluate the capabilities of Large Language Models (LLMs) is the focal point and hot topic in current LLMs research. Previous work has noted that due to the extremely high cost of iterative updates of LLMs, they are often unable to answer the latest dynamic questions well. To promote the improvement of Chinese LLMs' ability to answer dynamic questions, in this paper, we introduce CDQA, a Chinese Dynamic QA benchmark containing question-answer pairs related to the latest news on the Chinese Internet. We obtain high-quality data through a pipeline that combines humans and models, and carefully classify the samples according to the frequency of answer changes to facilitate a more fine-grained observation of LLMs' capabilities. We have also evaluated and analyzed mainstream and advanced Chinese LLMs on CDQA. Extensive experiments and valuable insights suggest that our proposed CDQA is challenging and worthy of more further study. We believe that the benchmark we provide will become one of the key data resources for improving LLMs' Chinese question-answering ability in the future.