mlsanityData to 7 Oct 2026

KILT: a Benchmark for Knowledge Intensive Language Tasks

arXiv 2009.02252 ↗First posted 4 Sep 2020cs.CL cs.AI cs.IRcited by 571

Not on the map. Why?

Cited by 181 in Retrieval-augmented generation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Challenging problems such as open-domain question answering, fact checking, slot filling and entity linking require access to large, external knowledge sources. While some models do well on individual tasks, developing general models is difficult as each task might require computationally expensive indexing of custom knowledge sources, in addition to dedicated infrastructure. To catalyze research on models that condition on specific information in large textual resources, we present a benchmark for knowledge-intensive language tasks (KILT). All tasks in KILT are grounded in the same snapshot of Wikipedia, reducing engineering turnaround through the re-use of components, as well as accelerating research into task-agnostic memory architectures. We test both task-specific and general baselines, evaluating downstream performance in addition to the ability of the models to provide provenance. We find that a shared dense vector index coupled with a seq2seq model is a strong baseline, outperforming more tailor-made approaches for fact checking, open-domain question answering and dialogue, and yielding competitive results on entity linking and slot filling, by generating disambiguated text. KILT data and code are available at https://github.com/facebookresearch/KILT.

Key facts. Cited by 571 papers in the map (data through 7 Oct 2026). First posted on arXiv 4 Sep 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 63; 2022: 77; 2023: 103; 2024: 134; 2025: 123; 2026: 71.

571 papers.

Sort:

By area:

  1. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  2. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  3. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  4. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  5. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
  6. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 893
  7. 2023In-Context Retrieval-Augmented Language Modelscited by 694
  8. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 691
  9. 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 616
  10. 2023Enabling Large Language Models to Generate Text with Citationscited by 584
  11. 2021Editing Factual Knowledge in Language Modelscited by 548
  12. 2023LaMP: When Large Language Models Meet Personalizationcited by 425
38 more
  1. 2024The Faiss librarycited by 370
  2. 2023Making Retrieval-Augmented Language Models Robust to Irrelevant Contextcited by 364
  3. 2024Retrieval-Augmented Generation for AI-Generated Content: A Surveycited by 352
  4. 2022One Embedder, Any Task: Instruction-Finetuned Text Embeddingscited by 329
  5. 2025R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learningcited by 322
  6. 2024Graph Retrieval-Augmented Generation: A Surveycited by 320
  7. 2024Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Modelscited by 301
  8. 2023Evaluating Verifiability in Generative Search Enginescited by 297
  9. 2022Generate rather than Retrieve: Large Language Models are Strong Context Generatorscited by 296
  10. 2023ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systemscited by 259
  11. 2022Prompting GPT-3 To Be Reliablecited by 258
  12. 2023RECOMP: Improving Retrieval-Augmented LMs with Compression and Selective Augmentationcited by 253
  13. 2021The GEM Benchmark: Natural Language Generation, its Evaluation and Metricscited by 239
  14. 2025Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactionscited by 236
  15. 2023Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graphcited by 235
  16. 2024Generative Representational Instruction Tuningcited by 227
  17. 2023Woodpecker: Hallucination Correction for Multimodal Large Language Modelscited by 223
  18. 2021Get Your Vitamin C! Robust Fact Verification with Contrastive Evidencecited by 222
  19. 2021Internet-Augmented Dialogue Generationcited by 217
  20. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 204
  21. 2022Factuality Enhanced Language Models for Open-Ended Text Generationcited by 200
  22. 2024RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMscited by 200
  23. 2021PAQ: 65 Million Probably-Asked Questions and What You Can Do With Themcited by 198
  24. 2023RA-DIT: Retrieval-Augmented Dual Instruction Tuningcited by 194
  25. 2023Knowledge Editing for Large Language Models: A Surveycited by 189
  26. 2021FeTaQA: Free-form Table Question Answeringcited by 174
  27. 2021Towards Continual Knowledge Learning of Language Modelscited by 172
  28. 2023Head-to-Tail: How Knowledgeable are Large Language Models (LLMs)? A.K.A. Will LLMs Replace Knowledge Graphs?cited by 167
  29. 2021Hurdles to Progress in Long-form Question Answeringcited by 166
  30. 2024FlashRAG: A Modular Toolkit for Efficient Retrieval-Augmented Generation Researchcited by 165
  31. 2021ExT5: Towards Extreme Multi-Task Scaling for Transfer Learningcited by 161
  32. 2024BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrievalcited by 153
  33. 2023UniMax: Fairer and more Effective Language Sampling for Large-Scale Multilingual Pretrainingcited by 147
  34. 2023Investigating the Factual Knowledge Boundary of Large Language Models with Retrieval Augmentationcited by 146
  35. 2024Evaluation of Retrieval-Augmented Generation: A Surveycited by 146
  36. 2021SituatedQA: Incorporating Extra-Linguistic Contexts into QAcited by 145
  37. 2022ELEVATER: A Benchmark and Toolkit for Evaluating Language-Augmented Visual Modelscited by 141
  38. 2025BrowseComp-ZH: Benchmarking Web Browsing Ability of Large Language Models in Chinesecited by 139

50 most cited of 571 All 571 (JSON)