# Explaining and Improving Model Behavior with k Nearest Neighbor Representations

arXiv: https://arxiv.org/abs/2010.09030 · HTML: https://mlsanity.com/paper/2010.09030/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Oct 2020.

## Built on by (most cited first, 27 of 27)

- [What Makes Good In-Context Examples for GPT-3?](https://mlsanity.com/paper/2101.06804/) (2021)
- [QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehension](https://mlsanity.com/paper/2107.12708/) (2021)
- [Combining Feature and Instance Attribution to Detect Artifacts](https://mlsanity.com/paper/2107.00323/) (2021)
- [An Empirical Comparison of Instance Attribution Methods for NLP](https://mlsanity.com/paper/2104.04128/) (2021)
- [Training Data Attribution via Approximate Unrolled Differentiation](https://mlsanity.com/paper/2405.12186/) (2024)
- [On Training Instance Selection for Few-Shot Neural Text Generation](https://mlsanity.com/paper/2107.03176/) (2021)
- [Towards Tracing Factual Knowledge in Language Models Back to the Training Data](https://mlsanity.com/paper/2205.11482/) (2022)
- [Interpreting Deep Learning Models in Natural Language Processing: A Review](https://mlsanity.com/paper/2110.10470/) (2021)
- [Cross-Domain Data Integration for Named Entity Disambiguation in Biomedical Text](https://mlsanity.com/paper/2110.08228/) (2021)
- [How do languages influence each other? Studying cross-lingual data sharing during LM fine-tuning](https://mlsanity.com/paper/2305.13286/) (2023)
- [Latent Concept-based Explanation of NLP Models](https://mlsanity.com/paper/2404.12545/) (2024)
- [Contrastive Error Attribution for Finetuned Language Models](https://mlsanity.com/paper/2212.10722/) (2022)
- [Improving and Diagnosing Knowledge-Based Visual Question Answering via Entity Enhanced Knowledge Injection](https://mlsanity.com/paper/2112.06888/) (2021)
- [Towards flexible perception with visual memory](https://mlsanity.com/paper/2408.08172/) (2024)
- [Enhancing Training Data Attribution with Representational Optimization](https://mlsanity.com/paper/2505.18513/) (2025)
- [Interpreting Embedding Spaces by Conceptualization](https://mlsanity.com/paper/2209.00445/) (2022)
- [Data-centric Prediction Explanation via Kernelized Stein Discrepancy](https://mlsanity.com/paper/2403.15576/) (2024)
- [Behavior of k-NN as an Instance-Based Explanation Method](https://mlsanity.com/paper/2109.06999/) (2021)
- [FASTTRACK: Fast and Accurate Fact Tracing for LLMs](https://mlsanity.com/paper/2404.15157/) (2024)
- [Wrapper Boxes: Faithful Attribution of Model Predictions to Training Data](https://mlsanity.com/paper/2311.08644/) (2023)
- [InfFeed: Influence Functions as a Feedback to Improve the Performance of Subjective Tasks](https://mlsanity.com/paper/2402.14702/) (2024)
- [Vector Quantized Latent Concepts: A Scalable Alternative to Clustering-Based Concept Discovery](https://mlsanity.com/paper/2602.02726/) (2026)
- [STRIDE: Training Data Attribution via Sparse Recovery from Subset Perturbations](https://mlsanity.com/paper/2606.05165/) (2026)
- [Separating Knowledge and Perception with Procedural Data](https://mlsanity.com/paper/2508.11697/) (2025)
- [Robust Nearest Neighbour Retrieval Using Targeted Manifold Manipulation](https://mlsanity.com/paper/2511.06261/) (2025)
- [Near-Exponential Convergence Rates for kNN Classification based on Boltzmann Margin](https://mlsanity.com/paper/2606.10361/) (2026)
- [Analysis of Quantized and Efficiently Adapted Protein Language Models](https://mlsanity.com/paper/2610.00665/) (2026)

## Abstract (arXiv)

Interpretability techniques in NLP have mainly focused on understanding individual predictions using attention visualization or gradient-based saliency maps over tokens. We propose using k nearest neighbor (kNN) representations to identify training examples responsible for a model's predictions and obtain a corpus-level understanding of the model's behavior. Apart from interpretability, we show that kNN representations are effective at uncovering learned spurious associations, identifying mislabeled examples, and improving the fine-tuned model's performance. We focus on Natural Language Inference (NLI) as a case study and experiment with multiple datasets. Our method deploys backoff to kNN for BERT and RoBERTa on examples with low model confidence without any update to the model parameters. Our results indicate that the kNN approach makes the finetuned model more robust to adversarial inputs.