mlsanityData to 7 Oct 2026

A Flexible Nadaraya-Watson Head Can Offer Explainable and Calibrated Classification

arXiv 2212.03411 ↗First posted 7 Dec 2022cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

In this paper, we empirically analyze a simple, non-learnable, and nonparametric Nadaraya-Watson (NW) prediction head that can be used with any neural network architecture. In the NW head, the prediction is a weighted average of labels from a support set. The weights are computed from distances between the query feature and support features. This is in contrast to the dominant approach of using a learnable classification head (e.g., a fully-connected layer) on the features, which can be challenging to interpret and can yield poorly calibrated predictions. Our empirical results on an array of computer vision tasks demonstrate that the NW head can yield better calibration with comparable accuracy compared to its parametric counterpart, particularly in data-limited settings. To further increase inference-time efficiency, we propose a simple approach that involves a clustering step run on the training set to create a relatively small distilled support set. Furthermore, we explore two means of interpretability/explainability that fall naturally from the NW head. The first is the label weights, and the second is our novel concept of the ``support influence function,'' which is an easy-to-compute metric that quantifies the influence of a support element on the prediction for a given query. As we demonstrate in our experiments, the influence function can allow the user to debug a trained model. We believe that the NW head is a flexible, interpretable, and highly useful building block that can be used in a range of applications.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 38 of its 57 references that are in the map. First posted on arXiv 7 Dec 2022.

Builds on

38 of 57 references.

References · 38

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2016Densely Connected Convolutional Networkscited by 6,691
  5. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  6. 2017On Calibration of Modern Neural Networkscited by 4,860
  7. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  8. 2016Matching Networks for One Shot Learningcited by 2,246
  9. 2016Identity Mappings in Deep Residual Networkscited by 1,960
  10. 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
  11. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
  12. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,421
26 more
  1. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  2. 2017Learning to Compare: Relation Network for Few-Shot Learningcited by 1,139
  3. 2021Perceiver: General Perception with Iterative Attentioncited by 1,089
  4. 2018This Looks Like That: Deep Learning for Interpretable Image Recognitioncited by 731
  5. 2015Deep Kernel Learningcited by 440
  6. 2019BBN: Bilateral-Branch Network with Cumulative Learning for Long-Tailed Visual Recognitioncited by 403
  7. 2021Revisiting the Calibration of Modern Neural Networkscited by 387
  8. 2017Residual Attention Network for Image Classificationcited by 385
  9. 2020Influence Functions in Deep Learning Are Fragilecited by 247
  10. 2020Dataset Meta-Learning from Kernel Ridge-Regressioncited by 230
  11. 2018Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learningcited by 203
  12. 2020Regularizing Class-wise Predictions via Self-knowledge Distillationcited by 144
  13. 2017Doubly Stochastic Variational Inference for Deep Gaussian Processescited by 143
  14. 2021End-to-End Training of Multi-Document Reader and Retriever for Open-Domain Question Answeringcited by 133
  15. 2021Interpretable Deep Learning: Interpretation, Interpretability, Trustworthiness, and Beyondcited by 116
  16. 2021Self-Attention Between Datapoints: Going Beyond Individual Input-Output Pairs in Deep Learningcited by 112
  17. 2019Analyzing and Improving Representations with the Soft Nearest Neighbor Losscited by 95
  18. 2018Interpreting CNNs via Decision Treescited by 89
  19. 2021Deep Learning for Instance Retrieval: A Surveycited by 70
  20. 2016One-shot Learning with Memory-Augmented Neural Networkscited by 60
  21. 2022Learning with Neighbor Consistency for Noisy Labelscited by 55
  22. 2020On Episodes, Prototypical Networks, and Few-shot Learningcited by 46
  23. 2016Deep Gaussian Processes for Regression using Approximate Expectation Propagationcited by 39
  24. 2022Visual correspondence-based explanations improve AI robustness and human-AI team accuracycited by 37
  25. 2018Diagnose like a Radiologist: Attention Guided Convolutional Neural Network for Thorax Disease Classificationcited by 36
  26. 2020TraDE: Transformers for Density Estimationcited by 18

Built on by

Cited by year

Citing papers per year: 2023: 3; 2024: 0; 2025: 1.

4 papers.

  1. 2023TabR: Tabular Deep Learning Meets Nearest Neighbors in 2023cited by 73
  2. 2023A Framework for Interpretability in Machine Learning for Medical Imagingcited by 9
  3. 2023Learning Invariant Representations with a Nonparametric Nadaraya-Watson Headcited by 1
  4. 2025SIC: Similarity-Based Interpretable Image Classification with Neural Networkscited by 0