mlsanityData to 7 Oct 2026

FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation

arXiv 2310.03214 ↗First posted 5 Oct 2023cs.CLcited by 308

Not on the map. Why?

Cited by 62 in Retrieval-augmented generation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Most large language models (LLMs) are trained once and never updated; thus, they lack the ability to dynamically adapt to our ever-changing world. In this work, we perform a detailed study of the factuality of LLM-generated text in the context of answering questions that test current world knowledge. Specifically, we introduce FreshQA, a novel dynamic QA benchmark encompassing a diverse range of question and answer types, including questions that require fast-changing world knowledge as well as questions with false premises that need to be debunked. We benchmark a diverse array of both closed and open-source LLMs under a two-mode evaluation procedure that allows us to measure both correctness and hallucination. Through human evaluations involving more than 50K judgments, we shed light on limitations of these models and demonstrate significant room for improvement: for instance, all models (regardless of model size) struggle on questions that involve fast-changing knowledge and false premises. Motivated by these results, we present FreshPrompt, a simple few-shot prompting method that substantially boosts the performance of an LLM on FreshQA by incorporating relevant and up-to-date information retrieved from a search engine into the prompt. Our experiments show that FreshPrompt outperforms both competing search engine-augmented prompting methods such as Self-Ask (Press et al., 2022) as well as commercial systems such as Perplexity.AI. Further analysis of FreshPrompt reveals that both the number of retrieved evidences and their order play a key role in influencing the correctness of LLM-generated answers. Additionally, instructing the LLM to generate concise and direct answers helps reduce hallucination compared to encouraging more verbose answers. To facilitate future work, we release FreshQA at github.com/freshllms/freshqa and commit to updating it at regular intervals.

Key facts. Cited by 308 papers in the map (data through 7 Oct 2026). Builds on 37 of its 41 references that are in the map. First posted on arXiv 5 Oct 2023.

Builds on

37 of 41 references.

References · 37

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  7. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  8. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  9. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  10. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  11. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  12. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
25 more
  1. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  2. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  3. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  4. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  5. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  6. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  7. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  8. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  9. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 895
  10. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 734
  11. 2019ELI5: Long Form Question Answeringcited by 567
  12. 2022Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLPcited by 316
  13. 2023Evaluating Verifiability in Generative Search Enginescited by 297
  14. 2022Generate rather than Retrieve: Large Language Models are Strong Context Generatorscited by 296
  15. 2022Teaching language models to support answers with verified quotescited by 290
  16. 2022RealTime QA: What's the Answer Right Now?cited by 239
  17. 2021A Dataset for Answering Time-Sensitive Questionscited by 150
  18. 2021SituatedQA: Incorporating Extra-Linguistic Contexts into QAcited by 145
  19. 2022Internet-augmented language models through few-shot prompting for open-domain question answeringcited by 131
  20. 2022StreamingQA: A Benchmark for Adaptation to New Knowledge over Time in Question Answering Modelscited by 123
  21. 2022CREPE: Open-Domain Question Answering with False Presuppositionscited by 70
  22. 2022Recitation-Augmented Language Modelscited by 65
  23. 2022(QA)2: Question Answering with Questionable Assumptionscited by 51
  24. 2023SAIL: Search-Augmented Instruction Learningcited by 33
  25. 2023IfQA: A Dataset for Open-domain Question Answering under Counterfactual Presuppositionscited by 25

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 14; 2024: 110; 2025: 105; 2026: 79.

308 papers.

Sort:

By area:

  1. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  2. 2025ToolRL: Reward is All Tool Learning Needscited by 342
  3. 2024Hallucination is Inevitable: An Innate Limitation of Large Language Modelscited by 329
  4. 2024Knowledge Conflicts for LLMs: A Surveycited by 274
  5. 2023RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Modelscited by 272
  6. 2024A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Modelscited by 266
  7. 2024RAFT: Adapting Language Model to Domain Specific RAGcited by 258
  8. 2024VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documentscited by 237
  9. 2025Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactionscited by 236
  10. 2024Tool Learning with Large Language Models: A Surveycited by 221
  11. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 204
  12. 2023Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Modelscited by 180
38 more
  1. 2024Fact, Fetch, and Reason: A Unified Evaluation of Retrieval-Augmented Generationcited by 158
  2. 2024A Comprehensive Study of Knowledge Editing for Large Language Modelscited by 157
  3. 2025Dynamic Cheatsheet: Test-Time Learning with Adaptive Memorycited by 146
  4. 2024RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generationcited by 144
  5. 2024Certifiably Robust RAG against Retrieval Corruptioncited by 144
  6. 2025BrowseComp-ZH: Benchmarking Web Browsing Ability of Large Language Models in Chinesecited by 139
  7. 2025AbstentionBench: Reasoning LLMs Fail on Unanswerable Questionscited by 129
  8. 2023Calibrated Language Models Must Hallucinatecited by 114
  9. 2024VERISCORE: Evaluating the factuality of verifiable claims in long-form text generationcited by 114
  10. 2024InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationalescited by 100
  11. 2025OverThink: Slowdown Attacks on Reasoning LLMscited by 94
  12. 2024From Matching to Generation: A Survey on Generative Information Retrievalcited by 89
  13. 2025HalluLens: LLM Hallucination Benchmarkcited by 88
  14. 2023Factcheck-Bench: Fine-Grained Evaluation Benchmark for Automatic Fact-checkerscited by 86
  15. 2024Fast Adversarial Attacks on Language Models In One GPU Minutecited by 85
  16. 2023Alleviating Hallucinations of Large Language Models through Induced Hallucinationscited by 78
  17. 2024Simple Is Effective: The Roles of Graphs and Large Language Models in Knowledge-Graph-Based Retrieval-Augmented Generationcited by 78
  18. 2024Foundational Autoraters: Taming Large Language Models for Better Automatic Evaluationcited by 72
  19. 2024Retrieval-Augmented Generation for Natural Language Processing: A Surveycited by 70
  20. 2024Sufficient Context: A New Lens on Retrieval Augmented Generation Systemscited by 69
  21. 2024Benchmarking Multimodal Retrieval Augmented Generation with Dynamic VQA Dataset and Self-adaptive Planning Agentcited by 65
  22. 2025SealQA: Raising the Bar for Reasoning in Search-Augmented Language Modelscited by 65
  23. 2024Factuality of Large Language Models: A Surveycited by 64
  24. 2024KnowAgent: Knowledge-Augmented Planning for LLM-Based Agentscited by 61
  25. 2024When Search Engine Services meet Large Language Models: Visions and Challengescited by 59
  26. 2025The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systemscited by 57
  27. 2025SMART: Self-Aware Agent for Tool Overuse Mitigationcited by 52
  28. 2024Position: What Can Large Language Models Tell Us about Time Series Analysiscited by 47
  29. 2024How Easy is It to Fool Your Multimodal LLMs? An Empirical Analysis on Deceptive Promptscited by 47
  30. 2024RaFe: Ranking Feedback Improves Query Rewriting for RAGcited by 47
  31. 2024MegaAgent: A Large-Scale Autonomous LLM-based Multi-Agent System Without Predefined SOPscited by 45
  32. 2025Large Language Models Hallucination: A Comprehensive Surveycited by 45
  33. 2025SimpleQA Verified: A Reliable Factuality Benchmark to Measure Parametric Knowledgecited by 44
  34. 2026DeepSearchQA: Bridging the Comprehensiveness Gap for Deep Research Agentscited by 44
  35. 2025Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluationcited by 41
  36. 2023Ever: Mitigating Hallucination in Large Language Models through Real-Time Verification and Rectificationcited by 38
  37. 2024Knowledge Boundary of Large Language Models: A Surveycited by 36
  38. 2024OmAgent: A Multi-modal Agent Framework for Complex Video Understanding with Task Divide-and-Conquercited by 34

50 most cited of 308 All 308 (JSON)