# kNN Prompting: Beyond-Context Learning with Calibration-Free Nearest Neighbor Inference

arXiv: https://arxiv.org/abs/2303.13824 · HTML: https://mlsanity.com/paper/2303.13824/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- Builds on 49 of its 65 references that are in the map.
- First posted on arXiv 24 Mar 2023.

## Builds on (most cited first, 30 of 49)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [Multitask Prompted Training Enables Zero-Shot Task Generalization](https://mlsanity.com/paper/2110.08207/) (2021)
- [Calibrate Before Use: Improving Few-Shot Performance of Language Models](https://mlsanity.com/paper/2102.09690/) (2021)
- [Improving language models by retrieving from trillions of tokens](https://mlsanity.com/paper/2112.04426/) (2021)
- [Scaling Language Models: Methods, Analysis & Insights from Training Gopher](https://mlsanity.com/paper/2112.11446/) (2021)
- [What Makes Good In-Context Examples for GPT-3?](https://mlsanity.com/paper/2101.06804/) (2021)
- [Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity](https://mlsanity.com/paper/2104.08786/) (2021)
- [How Can We Know What Language Models Know?](https://mlsanity.com/paper/1911.12543/) (2019)
- [Language (Technology) is Power: A Critical Survey of "Bias" in NLP](https://mlsanity.com/paper/2005.14050/) (2020)
- [Atlas: Few-shot Learning with Retrieval Augmented Language Models](https://mlsanity.com/paper/2208.03299/) (2022)
- [Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference](https://mlsanity.com/paper/2001.07676/) (2020)
- [An Explanation of In-context Learning as Implicit Bayesian Inference](https://mlsanity.com/paper/2111.02080/) (2021)
- [StereoSet: Measuring stereotypical bias in pretrained language models](https://mlsanity.com/paper/2004.09456/) (2020)
- [Generalization through Memorization: Nearest Neighbor Language Models](https://mlsanity.com/paper/1911.00172/) (2019)
- [Learning To Retrieve Prompts for In-Context Learning](https://mlsanity.com/paper/2112.08633/) (2021)
- [Deep Learning Scaling is Predictable, Empirically](https://mlsanity.com/paper/1712.00409/) (2017)

## Built on by (most cited first, 30 of 47)

- [Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations](https://mlsanity.com/paper/2310.06387/) (2023)
- [In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steering](https://mlsanity.com/paper/2311.06668/) (2023)
- [Unified Demonstration Retriever for In-Context Learning](https://mlsanity.com/paper/2305.04320/) (2023)
- [ExpertPrompting: Instructing Large Language Models to be Distinguished Experts](https://mlsanity.com/paper/2305.14688/) (2023)
- [Batch Calibration: Rethinking Calibration for In-Context Learning and Prompt Engineering](https://mlsanity.com/paper/2309.17249/) (2023)
- [On the Calibration of Large Language Models and Alignment](https://mlsanity.com/paper/2311.13240/) (2023)
- [Self-Knowledge Guided Retrieval Augmentation for Large Language Models](https://mlsanity.com/paper/2310.05002/) (2023)
- [Demystifying Chains, Trees, and Graphs of Thoughts](https://mlsanity.com/paper/2401.14295/) (2024)
- [Self-Verification Improves Few-Shot Clinical Information Extraction](https://mlsanity.com/paper/2306.00024/) (2023)
- [Forward-Backward Reasoning in Large Language Models for Mathematical Verification](https://mlsanity.com/paper/2308.07758/) (2023)
- [Which Examples to Annotate for In-Context Learning? Towards Effective and Efficient Selection](https://mlsanity.com/paper/2310.20046/) (2023)
- [Mixture of In-Context Prompters for Tabular PFNs](https://mlsanity.com/paper/2405.16156/) (2024)
- [Tree Prompting: Efficient Task Adaptation without Fine-Tuning](https://mlsanity.com/paper/2310.14034/) (2023)
- [Enhancing In-context Learning via Linear Probe Calibration](https://mlsanity.com/paper/2401.12406/) (2024)
- [InstructEval: Systematic Evaluation of Instruction Selection Methods](https://mlsanity.com/paper/2307.00259/) (2023)
- [Large Knowledge Model: Perspectives and Challenges](https://mlsanity.com/paper/2312.02706/) (2023)
- [Are Human-generated Demonstrations Necessary for In-context Learning?](https://mlsanity.com/paper/2309.14681/) (2023)
- [Few-shot Dialogue Strategy Learning for Motivational Interviewing via Inductive Reasoning](https://mlsanity.com/paper/2403.15737/) (2024)
- [Token-based Decision Criteria Are Suboptimal in In-context Learning](https://mlsanity.com/paper/2406.16535/) (2024)
- [Vocabulary-Defined Semantics: Latent Space Clustering for Improving In-Context Learning](https://mlsanity.com/paper/2401.16184/) (2024)
- [Dynamic Demonstrations Controller for In-Context Learning](https://mlsanity.com/paper/2310.00385/) (2023)
- [RecallM: An Adaptable Memory Mechanism with Temporal Understanding for Large Language Models](https://mlsanity.com/paper/2307.02738/) (2023)
- [No Train, all Gain: Self-Supervised Gradients Improve Deep Frozen Representations](https://mlsanity.com/paper/2407.10964/) (2024)
- [LAMPO: Large Language Models as Preference Machines for Few-shot Ordinal Classification](https://mlsanity.com/paper/2408.03359/) (2024)
- [Self-Instructed Derived Prompt Generation Meets In-Context Learning: Unlocking New Potential of Black-Box LLMs](https://mlsanity.com/paper/2409.01552/) (2024)
- [From Promising Capability to Pervasive Bias: Assessing Large Language Models for Emergency Department Triage](https://mlsanity.com/paper/2504.16273/) (2025)
- [A Study on Training and Developing Large Language Models for Behavior Tree Generation](https://mlsanity.com/paper/2401.08089/) (2024)
- [Attribute Structuring Improves LLM-Based Evaluation of Clinical Text Summaries](https://mlsanity.com/paper/2403.01002/) (2024)
- [Evaluating and Safeguarding the Adversarial Robustness of Retrieval-Based In-Context Learning](https://mlsanity.com/paper/2405.15984/) (2024)
- [TestNUC: Enhancing Test-Time Computing Approaches and Scaling through Neighboring Unlabeled Data Consistency](https://mlsanity.com/paper/2502.19163/) (2025)

## Abstract (arXiv)

In-Context Learning (ICL), which formulates target tasks as prompt completion conditioned on in-context demonstrations, has become the prevailing utilization of LLMs. In this paper, we first disclose an actual predicament for this typical usage that it can not scale up with training data due to context length restriction. Besides, existing works have shown that ICL also suffers from various biases and requires delicate calibration treatment. To address both challenges, we advocate a simple and effective solution, kNN Prompting, which first queries LLM with training data for distributed representations, then predicts test instances by simply referring to nearest neighbors. We conduct comprehensive experiments to demonstrate its two-fold superiority: 1) Calibration-Free: kNN Prompting does not directly align LLM output distribution with task-specific label space, instead leverages such distribution to align test and training instances. It significantly outperforms state-of-the-art calibration-based methods under comparable few-shot scenario. 2) Beyond-Context: kNN Prompting can further scale up effectively with as many training data as are available, continually bringing substantial improvements. The scaling trend holds across 10 orders of magnitude ranging from 2 shots to 1024 shots as well as different LLMs scales ranging from 0.8B to 30B. It successfully bridges data scaling into model scaling, and brings new potentials for the gradient-free paradigm of LLM deployment. Code is publicly available.