# DailyQA: A Benchmark to Evaluate Web Retrieval Augmented LLMs Based on Capturing Real-World Changes

arXiv: https://arxiv.org/abs/2505.17162 · HTML: https://mlsanity.com/paper/2505.17162/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation.
- Builds on 17 of its 24 references that are in the map.
- First posted on arXiv 22 May 2025.

## Placed in

- Retrieval-augmented generation / [RAG robustness and adaptive retrieval](https://mlsanity.com/rag/rag-robustness-and-adaptive-retrieval/)

## Builds on (most cited first, 17 of 17)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT](https://mlsanity.com/paper/2004.12832/) (2020)
- [DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks](https://mlsanity.com/paper/1704.04110/) (2017)
- [BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining](https://mlsanity.com/paper/2210.10341/) (2022)
- [FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation](https://mlsanity.com/paper/2310.03214/) (2023)
- [Time-Aware Language Models as Temporal Knowledge Bases](https://mlsanity.com/paper/2106.15110/) (2021)
- [A Dataset for Answering Time-Sensitive Questions](https://mlsanity.com/paper/2108.06314/) (2021)
- [UnSeenTimeQA: Time-Sensitive Question-Answering Beyond LLMs' Memorization](https://mlsanity.com/paper/2407.03525/) (2024)
- [MRAG: A Modular Retrieval Framework for Time-Sensitive Question Answering](https://mlsanity.com/paper/2412.15540/) (2024)
- [Deep Relevance Ranking Using Enhanced Document-Query Interactions](https://mlsanity.com/paper/1809.01682/) (2018)
- [Enhancing Temporal Sensitivity and Reasoning for Time-Sensitive Question Answering](https://mlsanity.com/paper/2409.16909/) (2024)
- [DREQ: Document Re-Ranking Using Entity-based Query Understanding](https://mlsanity.com/paper/2401.05939/) (2024)

## Built on by (most cited first, 3 of 3)

- [Stream RAG: Instant and Accurate Spoken Dialogue Systems with Streaming Tool Usage](https://mlsanity.com/paper/2510.02044/) (2025)
- [Look It Up: Analysing Internal Web Search Capabilities of Modern LLMs](https://mlsanity.com/paper/2511.18931/) (2025)
- [Benchmarking Real-Time Question Answering via Executable Code Workflows](https://mlsanity.com/paper/2604.16349/) (2026)

## Abstract (arXiv)

We propose DailyQA, an automatically updated dynamic dataset that updates questions weekly and contains answers to questions on any given date. DailyQA utilizes daily updates from Wikipedia revision logs to implement a fully automated pipeline of data filtering, query generation synthesis, quality checking, answer extraction, and query classification. The benchmark requires large language models (LLMs) to process and answer questions involving fast-changing factual data and covering multiple domains. We evaluate several open-source and closed-source LLMs using different RAG pipelines with web search augmentation. We compare the ability of different models to process time-sensitive web information and find that rerank of web retrieval results is critical. Our results indicate that LLMs still face significant challenges in handling frequently updated information, suggesting that DailyQA benchmarking provides valuable insights into the direction of progress for LLMs and RAG systems.