mlsanityData to 14 Oct 2026

PCNN: Probable-Class Nearest-Neighbor Explanations Improve Fine-Grained Image Classification Accuracy for AIs and Humans

arXiv 2308.13651 ↗First posted 25 Aug 2023cs.CV cs.HCcited by 3

Not on the map. Why?

Abstract (arXiv)

Nearest neighbors (NN) are traditionally used to compute final decisions, e.g., in Support Vector Machines or k-NN classifiers, and to provide users with explanations for the model's decision. In this paper, we show a novel utility of nearest neighbors: To improve predictions of a frozen, pretrained image classifier C. We leverage an image comparator S that (1) compares the input image with NN images from the top-K most probable classes given by C; and (2) uses scores from S to weight the confidence scores of C to refine predictions. Our method consistently improves fine-grained image classification accuracy on CUB-200, Cars-196, and Dogs-120. Also, a human study finds that showing users our probable-class nearest neighbors (PCNN) reduces over-reliance on AI, thus improving their decision accuracy over prior work which only shows only the most-probable (top-1) class examples.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 31 of its 54 references that are in the map. First posted on arXiv 25 Aug 2023.

Builds on

31 of 54 references.

References · 31

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  4. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  5. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  6. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  7. 2017Billion-scale similarity search with GPUscited by 2,497
  8. 2021CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classificationcited by 806
  9. 2018This Looks Like That: Deep Learning for Interpretable Image Recognitioncited by 737
  10. 2020Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performancecited by 399
  11. 2021TrivialAugment: Tuning-free Yet State-of-the-Art Data Augmentationcited by 227
  12. 2020Neural Prototype Trees for Interpretable Fine-grained Image Recognitioncited by 214
19 more
  1. 2018Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learningcited by 207
  2. 2022CRAFT: Concept Recursive Activation FacTorization for Explainabilitycited by 178
  3. 2021Deformable ProtoPNet: An Interpretable Image Classifier Using Deformable Prototypescited by 120
  4. 2021HIVE: Evaluating the Human Interpretability of Visual Explanationscited by 105
  5. 2018Learning to Navigate for Fine-grained Classificationcited by 103
  6. 2021What I Cannot Predict, I Do Not Understand: A Human-Centered Evaluation Framework for Explainability Methodscited by 98
  7. 2021Interpretable Image Classification with Differentiable Prototypes Assignmentcited by 97
  8. 2020ProtoPShare: Prototype Sharing for Interpretable Image Classification and Similarity Discoverycited by 89
  9. 2021The effectiveness of feature attribution methods and its correlation with automatic evaluation scorescited by 88
  10. 2023Appropriate Reliance on AI Advice: Conceptualization and the Effect of Explanationscited by 83
  11. 2023Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanationscited by 73
  12. 2023In Search of Verifiability: Explanations Rarely Enable Complementary Performance in AI-Advised Decision Makingcited by 50
  13. 2020PARADE: Passage Representation Aggregation for Document Rerankingcited by 44
  14. 2022Visual correspondence-based explanations improve AI robustness and human-AI team accuracycited by 39
  15. 2023Learning Support and Trivial Prototypes for Interpretable Image Classificationcited by 38
  16. 2022Proto2Proto: Can you recognize the car, the way I do?cited by 23
  17. 2021DeepFace-EMD: Re-ranking Using Patch-wise Earth Mover's Distance Improves Out-Of-Distribution Face Identificationcited by 20
  18. 2022Machine Explanations and Human Understandingcited by 20
  19. 2023Learning Human-Compatible Representations for Case-Based Decision Supportcited by 4

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 1.

3 papers.

  1. 2024Interpretable LLM-based Table Question Answeringcited by 18
  2. 2024Allowing humans to interactively guide machines where to look does not always improve human-AI team's classification accuracycited by 5
  3. 2025Conversational Explanations: Discussing Explainable AI with Non-AI Expertscited by 1