# BERT-kNN: Adding a kNN Search Component to Pretrained Language Models for Better QA

arXiv: https://arxiv.org/abs/2005.00766 · HTML: https://mlsanity.com/paper/2005.00766/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 May 2020.

## Built on by (most cited first, 30 of 36)

- [What Makes Good In-Context Examples for GPT-3?](https://mlsanity.com/paper/2101.06804/) (2021)
- [PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models](https://mlsanity.com/paper/2402.07867/) (2024)
- [Multilingual LAMA: Investigating Knowledge in Multilingual Pretrained Language Models](https://mlsanity.com/paper/2102.00894/) (2021)
- [Knowledgeable or Educated Guess? Revisiting Language Models as Knowledge Bases](https://mlsanity.com/paper/2106.09231/) (2021)
- [Adaptive Semiparametric Language Models](https://mlsanity.com/paper/2102.02557/) (2021)
- [Adaptive Nearest Neighbor Machine Translation](https://mlsanity.com/paper/2105.13022/) (2021)
- [Retrieval-Enhanced Machine Learning](https://mlsanity.com/paper/2205.01230/) (2022)
- [Nonparametric Masked Language Modeling](https://mlsanity.com/paper/2212.01349/) (2022)
- [Decoupling Knowledge from Memorization: Retrieval-augmented Prompt Learning](https://mlsanity.com/paper/2205.14704/) (2022)
- [ConvFiT: Conversational Fine-Tuning of Pretrained Language Models](https://mlsanity.com/paper/2109.10126/) (2021)
- [DREAM: Improving Situational QA by First Elaborating the Situation](https://mlsanity.com/paper/2112.08656/) (2021)
- [You can't pick your neighbors, or can you? When and how to rely on retrieval in the kNN-LM](https://mlsanity.com/paper/2210.15859/) (2022)
- [Relation Extraction as Open-book Examination: Retrieval-enhanced Prompt Tuning](https://mlsanity.com/paper/2205.02355/) (2022)
- [Reasoning Through Memorization: Nearest Neighbor Knowledge Graph Embeddings](https://mlsanity.com/paper/2201.05575/) (2022)
- [Pre-training Limited Memory Language Models with Internal and External Knowledge](https://mlsanity.com/paper/2505.15962/) (2025)
- [Static Embeddings as Efficient Knowledge Bases?](https://mlsanity.com/paper/2104.07094/) (2021)
- [Retrieval-Enhanced Machine Learning: Synthesis and Opportunities](https://mlsanity.com/paper/2407.12982/) (2024)
- [Defending Against Knowledge Poisoning Attacks During Retrieval-Augmented Generation](https://mlsanity.com/paper/2508.02835/) (2025)
- [Not Far Away, Not So Close: Sample Efficient Nearest Neighbour Data Augmentation via MiniMax](https://mlsanity.com/paper/2105.13608/) (2021)
- [Legal Question-Answering in the Indian Context: Efficacy, Challenges, and Potential of Modern AI Models](https://mlsanity.com/paper/2309.14735/) (2023)
- [KNN-BERT: Fine-Tuning Pre-Trained Models with KNN Classifier](https://mlsanity.com/paper/2110.02523/) (2021)
- [Retrieval-Enhanced Visual Prompt Learning for Few-shot Classification](https://mlsanity.com/paper/2306.02243/) (2023)
- [ARAIDA: Analogical Reasoning-Augmented Interactive Data Annotation](https://mlsanity.com/paper/2405.11912/) (2024)
- [Error-Robust Retrieval for Chinese Spelling Check](https://mlsanity.com/paper/2211.07843/) (2022)
- [Mitigating Catastrophic Forgetting in Task-Incremental Continual Learning with Adaptive Classification Criterion](https://mlsanity.com/paper/2305.12270/) (2023)
- [A Neighbourhood Framework for Resource-Lean Content Flagging](https://mlsanity.com/paper/2103.17055/) (2021)
- [WikiOmnia: generative QA corpus on the whole Russian Wikipedia](https://mlsanity.com/paper/2204.08009/) (2022)
- [Augmentation with Projection: Towards an Effective and Efficient Data Augmentation Paradigm for Distillation](https://mlsanity.com/paper/2210.11768/) (2022)
- [Machine Reading Comprehension using Case-based Reasoning](https://mlsanity.com/paper/2305.14815/) (2023)
- [RSpell: Retrieval-augmented Framework for Domain Adaptive Chinese Spelling Check](https://mlsanity.com/paper/2308.08176/) (2023)

## Abstract (arXiv)

Khandelwal et al. (2020) use a k-nearest-neighbor (kNN) component to improve language model performance. We show that this idea is beneficial for open-domain question answering (QA). To improve the recall of facts encountered during training, we combine BERT (Devlin et al., 2019) with a traditional information retrieval step (IR) and a kNN search over a large datastore of an embedded text collection. Our contributions are as follows: i) BERT-kNN outperforms BERT on cloze-style QA by large margins without any further training. ii) We show that BERT often identifies the correct response category (e.g., US city), but only kNN recovers the factually correct answer (e.g., "Miami"). iii) Compared to BERT, BERT-kNN excels for rare facts. iv) BERT-kNN can easily handle facts not covered by BERT's training set, e.g., recent events.