# kNN-Prompt: Nearest Neighbor Zero-Shot Inference

arXiv: https://arxiv.org/abs/2205.13792 · HTML: https://mlsanity.com/paper/2205.13792/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Placed in Generation and context integration, one of 15 directions in Retrieval-augmented generation.
- Builds on 27 of its 39 references that are in the map.
- First posted on arXiv 27 May 2022.

## Placed in

- Retrieval-augmented generation / [Generation and context integration](https://mlsanity.com/rag/generation-and-context-integration/)

## Builds on (most cited first, 27 of 27)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge](https://mlsanity.com/paper/1811.00937/) (2018)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [Calibrate Before Use: Improving Few-Shot Performance of Language Models](https://mlsanity.com/paper/2102.09690/) (2021)
- [Improving language models by retrieving from trillions of tokens](https://mlsanity.com/paper/2112.04426/) (2021)
- [Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering](https://mlsanity.com/paper/2007.01282/) (2020)
- [What Makes Good In-Context Examples for GPT-3?](https://mlsanity.com/paper/2101.06804/) (2021)
- [Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity](https://mlsanity.com/paper/2104.08786/) (2021)
- [ConceptNet 5.5: An Open Multilingual Graph of General Knowledge](https://mlsanity.com/paper/1612.03975/) (2016)
- [Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference](https://mlsanity.com/paper/2001.07676/) (2020)
- [Generalization through Memorization: Nearest Neighbor Language Models](https://mlsanity.com/paper/1911.00172/) (2019)
- [Learning To Retrieve Prompts for In-Context Learning](https://mlsanity.com/paper/2112.08633/) (2021)
- [Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering](https://mlsanity.com/paper/1602.01585/) (2016)
- [MetaICL: Learning to Learn In Context](https://mlsanity.com/paper/2110.15943/) (2021)
- [Surface Form Competition: Why the Highest Probability Answer Isn't Always Right](https://mlsanity.com/paper/2104.08315/) (2021)
- [Nearest Neighbor Machine Translation](https://mlsanity.com/paper/2010.00710/) (2020)
- [Noisy Channel Language Model Prompting for Few-Shot Text Classification](https://mlsanity.com/paper/2108.04106/) (2021)
- [Adapting Language Models for Zero-shot Learning by Meta-tuning on Dataset and Prompt Collections](https://mlsanity.com/paper/2104.04670/) (2021)
- [Efficient Nearest Neighbor Language Models](https://mlsanity.com/paper/2109.04212/) (2021)

## Built on by (most cited first, 27 of 27)

- [REPLUG: Retrieval-Augmented Black-Box Language Models](https://mlsanity.com/paper/2301.12652/) (2023)
- [Text Classification via Large Language Models](https://mlsanity.com/paper/2305.08377/) (2023)
- [PromptCap: Prompt-Guided Task-Aware Image Captioning](https://mlsanity.com/paper/2211.09699/) (2022)
- [Reliable, Adaptable, and Attributable Language Models with Retrieval](https://mlsanity.com/paper/2403.03187/) (2024)
- [SILO Language Models: Isolating Legal Risk In a Nonparametric Datastore](https://mlsanity.com/paper/2308.04430/) (2023)
- [Self-Knowledge Guided Retrieval Augmentation for Large Language Models](https://mlsanity.com/paper/2310.05002/) (2023)
- [TIARA: Multi-grained Retrieval for Robust Question Answering over Large Knowledge Bases](https://mlsanity.com/paper/2210.12925/) (2022)
- [kNN Prompting: Beyond-Context Learning with Calibration-Free Nearest Neighbor Inference](https://mlsanity.com/paper/2303.13824/) (2023)
- [Knowledge Card: Filling LLMs' Knowledge Gaps with Plug-in Specialized Language Models](https://mlsanity.com/paper/2305.09955/) (2023)
- [Nonparametric Masked Language Modeling](https://mlsanity.com/paper/2212.01349/) (2022)
- [RAVEN: In-Context Learning with Retrieval-Augmented Encoder-Decoder Language Models](https://mlsanity.com/paper/2308.07922/) (2023)
- [How Do Large Language Models Capture the Ever-changing World Knowledge? A Review of Recent Advances](https://mlsanity.com/paper/2310.07343/) (2023)
- [Data-Efficient Finetuning Using Cross-Task Nearest Neighbors](https://mlsanity.com/paper/2212.00196/) (2022)
- [ReGen: Zero-Shot Text Classification via Training Data Generation with Progressive Dense Retrieval](https://mlsanity.com/paper/2305.10703/) (2023)
- [Guess the Instruction! Flipped Learning Makes Language Models Stronger Zero-Shot Learners](https://mlsanity.com/paper/2210.02969/) (2022)
- [MLP Memory: A Retriever-Pretrained Memory for Large Language Models](https://mlsanity.com/paper/2508.01832/) (2025)
- [Memory Decoder: A Pretrained, Plug-and-Play Memory for Large Language Models](https://mlsanity.com/paper/2508.09874/) (2025)
- [kNN-Adapter: Efficient Domain Adaptation for Black-Box Language Models](https://mlsanity.com/paper/2302.10879/) (2023)
- [JoPA:Explaining Large Language Model's Generation via Joint Prompt Attribution](https://mlsanity.com/paper/2405.20404/) (2024)
- [Evaluating and Safeguarding the Adversarial Robustness of Retrieval-Based In-Context Learning](https://mlsanity.com/paper/2405.15984/) (2024)
- [Superposed Decoding: Multiple Generations from a Single Autoregressive Inference Pass](https://mlsanity.com/paper/2405.18400/) (2024)
- [Language Modelling Approaches to Adaptive Machine Translation](https://mlsanity.com/paper/2401.14559/) (2024)
- [Efficiently Enhancing Zero-Shot Performance of Instruction Following Model via Retrieval of Soft Prompt](https://mlsanity.com/paper/2210.03029/) (2022)
- [CELDA: Leveraging Black-box Language Model as Enhanced Classifier without Labels](https://mlsanity.com/paper/2306.02693/) (2023)
- [R4: Reinforced Retriever-Reorder-Responder for Retrieval-Augmented Large Language Models](https://mlsanity.com/paper/2405.02659/) (2024)
- [Revisiting k-NN for Fine-tuning Pre-trained Language Models](https://mlsanity.com/paper/2304.09058/) (2023)
- [DelvePO: Direction-Guided Self-Evolving Framework for Flexible Prompt Optimization](https://mlsanity.com/paper/2510.18257/) (2025)

## Abstract (arXiv)

Retrieval-augmented language models (LMs) use non-parametric memory to substantially outperform their non-retrieval counterparts on perplexity-based evaluations, but it is an open question whether they achieve similar gains in few- and zero-shot end-task accuracy. We extensively study one such model, the k-nearest neighbor LM (kNN-LM), showing that the gains marginally transfer. The main challenge is to achieve coverage of the verbalizer tokens that define the different end-task class labels. To address this challenge, we also introduce kNN-Prompt, a simple and effective kNN-LM with automatically expanded fuzzy verbalizers (e.g. to expand terrible to also include silly and other task-specific synonyms for sentiment classification). Across nine diverse end-tasks, using kNN-Prompt with GPT-2 large yields significant performance boosts over strong zero-shot baselines (13.4% absolute improvement over the base LM on average). We also show that other advantages of non-parametric augmentation hold for end tasks; kNN-Prompt is effective for domain adaptation with no further training, and gains increase with the size of the retrieval model.