mlsanityData to 7 Oct 2026

Analyzing and Improving Representations with the Soft Nearest Neighbor Loss

arXiv 1902.01889 ↗First posted 5 Feb 2019stat.ML cs.LGcited by 95

Not on the map. Why?

Abstract (arXiv)

We explore and expand the textitSoft Nearest Neighbor Loss to measure the textitentanglement of class manifolds in representation space: i.e., how close pairs of points from the same class are relative to pairs of points from different classes. We demonstrate several use cases of the loss. As an analytical tool, it provides insights into the evolution of class similarity structures during learning. Surprisingly, we find that textitmaximizing the entanglement of representations of different classes in the hidden layers is beneficial for discrimination in the final layer, possibly because it encourages representations to identify class-independent similarity structures. Maximizing the soft nearest neighbor loss in the hidden layers leads not only to improved generalization but also to better-calibrated estimates of uncertainty on outlier data. Data that is not from the training distribution can be recognized by observing that in the hidden layers, it has fewer than the normal number of neighbors from the predicted class.

Key facts. Cited by 95 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Feb 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 14; 2022: 18; 2023: 21; 2024: 21; 2025: 10; 2026: 11.

95 papers.

Sort:

By area:

  1. 2021With a Little Help from My Friends: Nearest-Neighbor Contrastive Learning of Visual Representationscited by 324
  2. 2021Dissecting Supervised Contrastive Learningcited by 64
  3. 2022Contrastive Boundary Learning for Point Cloud Segmentationcited by 56
  4. 2022Neural Collapse with Normalized Features: A Geometric Analysis over the Riemannian Manifoldcited by 50
  5. 2022Mitigating Data Sparsity for Short Text Topic Modeling by Topic-Semantic Contrastive Learningcited by 38
  6. 2024Revisiting Nearest Neighbor for Tabular Data: A Deep Tabular Baseline Two Decades Latercited by 28
  7. 2022Beyond Just Vision: A Review on Self-Supervised Representation Learning on Multimodal and Temporal Datacited by 27
  8. 2025It Takes Two: Your GRPO Is Secretly DPOcited by 25
  9. 2024Adapting to Distribution Shift by Visual Domain Prompt Generationcited by 23
  10. 2022On the Difficulty of Defending Self-Supervised Learning against Model Extractioncited by 22
  11. 2021ReMeDi: Resources for Multi-domain, Multi-service, Medical Dialoguescited by 20
  12. 2024Separating the "Chirp" from the "Chat": Self-supervised Visual Grounding of Sound and Languagecited by 17
38 more
  1. 2021A Closer Look at Few-Shot Video Classification: A New Baseline and Benchmarkcited by 16
  2. 2021Learning Debiased and Disentangled Representations for Semantic Segmentationcited by 16
  3. 2024Modeling Dynamic Topics in Chain-Free Fashion by Evolution-Tracking Contrastive Learning and Unassociated Word Exclusioncited by 16
  4. 2025Neighbor GRPO: Contrastive ODE Policy Optimization Aligns Flow Modelscited by 16
  5. 2023Quality-aware Pre-trained Models for Blind Image Quality Assessmentcited by 15
  6. 2023Toward Fairness Through Fair Multi-Exit Framework for Dermatological Disease Diagnosiscited by 14
  7. 2023Uncovering Intermediate Variables in Transformers using Circuit Probingcited by 12
  8. 2023Contrastive Learning for Enhancing Robust Scene Transfer in Vision-based Agile Flightcited by 11
  9. 2024Symbol as Points: Panoptic Symbol Spotting via Point-based Representationcited by 11
  10. 2025Syntriever: How to Train Your Retriever with Synthetic Data from LLMscited by 11
  11. 2024Achieve Fairness without Demographics for Dermatological Disease Diagnosiscited by 10
  12. 2021Fool Me Once: Robust Selective Segmentation via Out-of-Distribution Detection with Contrastive Learningcited by 9
  13. 2023Leveraging Multi-lingual Positive Instances in Contrastive Learning to Improve Sentence Embeddingcited by 9
  14. 2024Achieving Fairness Through Channel Pruning for Dermatological Disease Diagnosiscited by 9
  15. 2022Logit Normalization for Long-tail Object Detectioncited by 8
  16. 2023Identity Encoder for Personalized Diffusioncited by 8
  17. 2023Bucks for Buckets (B4B): Active Defenses Against Stealing Encoderscited by 8
  18. 2024End-to-End Training Induces Information Bottleneck through Layer-Role Differentiation: A Comparative Analysis with Layer-wise Trainingcited by 8
  19. 2024Learning Data Association for Multi-Object Tracking using Only Coordinatescited by 8
  20. 2023SCoRe: Submodular Combinatorial Representation Learningcited by 7
  21. 2023Inversion dynamics of class manifolds in deep learning reveals tradeoffs underlying generalisationcited by 6
  22. 2023Deep Classifier Mimicry without Data Accesscited by 6
  23. 2022Empirical Evaluation and Theoretical Analysis for Representation Learning: A Surveycited by 5
  24. 2023Contrastive Initial State Buffer for Reinforcement Learningcited by 5
  25. 2023Unsupervised Pretraining for Fact Verification by Language Model Distillationcited by 5
  26. 2026CLIPO: Contrastive Learning in Policy Optimization Generalizes RLVRcited by 5
  27. 2021SoundCLR: Contrastive Learning of Representations For Improved Environmental Sound Classificationcited by 4
  28. 2021Privacy Inference Attacks and Defenses in Cloud-based Deep Neural Network: A Surveycited by 4
  29. 2021ASK: Adversarial Soft k-Nearest Neighbor Attack and Defensecited by 4
  30. 2022A Flexible Nadaraya-Watson Head Can Offer Explainable and Calibrated Classificationcited by 4
  31. 2024Towards Robust Vision Transformer via Masked Adaptive Ensemblecited by 4
  32. 2022Semantic-based Pre-training for Dialogue Understandingcited by 3
  33. 2023Numerical Data Imputation for Multimodal Data Sets: A Probabilistic Nearest-Neighbor Kernel Density Approachcited by 3
  34. 2024Unsupervised Contrastive Learning for Robust RF Device Fingerprinting Under Time-Domain Shiftcited by 3
  35. 2022Incremental Few-Shot Learning via Implanting and Compressingcited by 2
  36. 2022Supervised Contrastive CSI Representation Learning for Massive MIMO Positioningcited by 2
  37. 2022p-DkNN: Out-of-Distribution Detection Through Statistical Testing of Deep Representationscited by 2
  38. 2022Improving Meta-Learning Generalization with Activation-Based Early-Stoppingcited by 2

50 most cited of 95 All 95 (JSON)