# Improving Generalization via Scalable Neighborhood Component Analysis

arXiv: https://arxiv.org/abs/1808.04699 · HTML: https://mlsanity.com/paper/1808.04699/

## Key facts

- Cited by 53 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Aug 2018.

## Built on by (most cited first, 30 of 53)

- [With a Little Help from My Friends: Nearest-Neighbor Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2104.14548/) (2021)
- [CReST: A Class-Rebalancing Self-Training Framework for Imbalanced Semi-Supervised Learning](https://mlsanity.com/paper/2102.09559/) (2021)
- [Rethinking Semantic Segmentation: A Prototype View](https://mlsanity.com/paper/2203.15102/) (2022)
- [Constrained Few-shot Class-incremental Learning](https://mlsanity.com/paper/2203.16588/) (2022)
- [Training Vision Transformers for Image Retrieval](https://mlsanity.com/paper/2102.05644/) (2021)
- [Visual Recognition with Deep Nearest Centroids](https://mlsanity.com/paper/2209.07383/) (2022)
- [Learning Cross-modal Contrastive Features for Video Domain Adaptation](https://mlsanity.com/paper/2108.11974/) (2021)
- [Towards In-context Scene Understanding](https://mlsanity.com/paper/2306.01667/) (2023)
- [Universal Weakly Supervised Segmentation by Pixel-to-Segment Contrastive Learning](https://mlsanity.com/paper/2105.00957/) (2021)
- [Know Your Self-supervised Learning: A Survey on Image-based Generative and Discriminative Training](https://mlsanity.com/paper/2305.13689/) (2023)
- [Rethinking Supervised Pre-training for Better Downstream Transferring](https://mlsanity.com/paper/2110.06014/) (2021)
- [Revisiting Nearest Neighbor for Tabular Data: A Deep Tabular Baseline Two Decades Later](https://mlsanity.com/paper/2407.03257/) (2024)
- [Complementary Pseudo Labels For Unsupervised Domain Adaptation On Person Re-identification](https://mlsanity.com/paper/2101.12521/) (2021)
- [Quantum Self-Supervised Learning](https://mlsanity.com/paper/2103.14653/) (2021)
- [Self-Supervision Can Be a Good Few-Shot Learner](https://mlsanity.com/paper/2207.09176/) (2022)
- [WiOpen: A Robust Wi-Fi-based Open-set Gesture Recognition Framework](https://mlsanity.com/paper/2402.00822/) (2024)
- [RecursiveMix: Mixed Learning with History](https://mlsanity.com/paper/2203.06844/) (2022)
- [COARSE3D: Class-Prototypes for Contrastive Learning in Weakly-Supervised 3D Point Cloud Segmentation](https://mlsanity.com/paper/2210.01784/) (2022)
- [On the Importance of Distractors for Few-Shot Classification](https://mlsanity.com/paper/2109.09883/) (2021)
- [Supervised Contrastive Learning with Hard Negative Samples](https://mlsanity.com/paper/2209.00078/) (2022)
- [iCAR: Bridging Image Classification and Image-text Alignment for Visual Recognition](https://mlsanity.com/paper/2204.10760/) (2022)
- [Learning with Explicit Shape Priors for Medical Image Segmentation](https://mlsanity.com/paper/2303.17967/) (2023)
- [Fast and Accurate Transferability Measurement by Evaluating Intra-class Feature Variance](https://mlsanity.com/paper/2308.05986/) (2023)
- [A Survey on Wi-Fi Sensing Generalizability: Taxonomy, Techniques, Datasets, and Future Research Prospects](https://mlsanity.com/paper/2503.08008/) (2025)
- [Positive Unlabeled Contrastive Learning](https://mlsanity.com/paper/2206.01206/) (2022)
- [GeNIe: Generative Hard Negative Images Through Diffusion](https://mlsanity.com/paper/2312.02548/) (2023)
- [Constrained Mean Shift Using Distant Yet Related Neighbors for Representation Learning](https://mlsanity.com/paper/2112.04607/) (2021)
- [Rethinking Nearest Neighbors for Visual Classification](https://mlsanity.com/paper/2112.08459/) (2021)
- [Zero-Shot Aspect-Based Sentiment Analysis](https://mlsanity.com/paper/2202.01924/) (2022)
- [Maximally Compact and Separated Features with Regular Polytope Networks](https://mlsanity.com/paper/2301.06116/) (2023)

## Abstract (arXiv)

Current major approaches to visual recognition follow an end-to-end formulation that classifies an input image into one of the pre-determined set of semantic categories. Parametric softmax classifiers are a common choice for such a closed world with fixed categories, especially when big labeled data is available during training. However, this becomes problematic for open-set scenarios where new categories are encountered with very few examples for learning a generalizable parametric classifier. We adopt a non-parametric approach for visual recognition by optimizing feature embeddings instead of parametric classifiers. We use a deep neural network to learn the visual feature that preserves the neighborhood structure in the semantic space, based on the Neighborhood Component Analysis (NCA) criterion. Limited by its computational bottlenecks, we devise a mechanism to use augmented memory to scale NCA for large datasets and very deep networks. Our experiments deliver not only remarkable performance on ImageNet classification for such a simple non-parametric method, but most importantly a more generalizable feature representation for sub-category discovery and few-shot recognition.