# Test-Time Adaptation via Self-Training with Nearest Neighbor Information

arXiv: https://arxiv.org/abs/2207.10792 · HTML: https://mlsanity.com/paper/2207.10792/

## Key facts

- Cited by 64 papers in the map (data through 14 Oct 2026).
- Builds on 48 of its 78 references that are in the map.
- First posted on arXiv 8 Jul 2022.

## Builds on (most cited first, 30 of 48)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [MixMatch: A Holistic Approach to Semi-Supervised Learning](https://mlsanity.com/paper/1905.02249/) (2019)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation](https://mlsanity.com/paper/1607.01719/) (2016)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [Temporal Ensembling for Semi-Supervised Learning](https://mlsanity.com/paper/1610.02242/) (2016)
- [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020)
- [Deep Clustering for Unsupervised Learning of Visual Features](https://mlsanity.com/paper/1807.05520/) (2018)
- [Deeper, Broader and Artier Domain Generalization](https://mlsanity.com/paper/1710.03077/) (2017)
- [Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2002.08546/) (2020)
- [Generalizing to Unseen Domains: A Survey on Domain Generalization](https://mlsanity.com/paper/2103.03097/) (2021)
- [Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution](https://mlsanity.com/paper/2202.10054/) (2022)
- [Recognition in Terra Incognita](https://mlsanity.com/paper/1807.04975/) (2018)
- [Self-labelling via simultaneous clustering and representation learning](https://mlsanity.com/paper/1911.05371/) (2019)
- [Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learning](https://mlsanity.com/paper/1908.02983/) (2019)
- [Measuring Robustness to Natural Distribution Shifts in Image Classification](https://mlsanity.com/paper/2007.00644/) (2020)
- [Improving robustness against common corruptions by covariate shift adaptation](https://mlsanity.com/paper/2006.16971/) (2020)
- [TADAM: Task dependent adaptive metric for improved few-shot learning](https://mlsanity.com/paper/1805.10123/) (2018)
- [BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learning](https://mlsanity.com/paper/2002.06715/) (2020)
- [Calibrating Deep Neural Networks using Focal Loss](https://mlsanity.com/paper/2002.09437/) (2020)
- [Quantifying Generalization in Reinforcement Learning](https://mlsanity.com/paper/1812.02341/) (2018)

## Built on by (most cited first, 30 of 64)

- [A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts](https://mlsanity.com/paper/2303.15361/) (2023)
- [Robust Test-Time Adaptation in Dynamic Scenarios](https://mlsanity.com/paper/2303.13899/) (2023)
- [In Search of Lost Online Test-time Adaptation: A Survey](https://mlsanity.com/paper/2310.20199/) (2023)
- [Feature Alignment and Uniformity for Test Time Adaptation](https://mlsanity.com/paper/2303.10902/) (2023)
- [TEA: Test-time Energy Adaptation](https://mlsanity.com/paper/2311.14402/) (2023)
- [Historical Test-time Prompt Tuning for Vision Foundation Models](https://mlsanity.com/paper/2410.20346/) (2024)
- [Bayesian Test-Time Adaptation for Vision-Language Models](https://mlsanity.com/paper/2503.09248/) (2025)
- [Contextual Document Embeddings](https://mlsanity.com/paper/2410.02525/) (2024)
- [Optimization-Free Test-Time Adaptation for Cross-Person Activity Recognition](https://mlsanity.com/paper/2310.18562/) (2023)
- [What, How, and When Should Object Detectors Update in Continually Changing Test Domains?](https://mlsanity.com/paper/2312.08875/) (2023)
- [AUTO: Adaptive Outlier Optimization for Test-Time OOD Detection](https://mlsanity.com/paper/2303.12267/) (2023)
- [Learning with Calibration: Exploring Test-Time Computing of Spatio-Temporal Forecasting](https://mlsanity.com/paper/2506.00635/) (2025)
- [Jailbreaking the Non-Transferable Barrier via Test-Time Data Disguising](https://mlsanity.com/paper/2503.17198/) (2025)
- [Decoupled Prototype Learning for Reliable Test-Time Adaptation](https://mlsanity.com/paper/2401.08703/) (2024)
- [A Layer Selection Approach to Test Time Adaptation](https://mlsanity.com/paper/2404.03784/) (2024)
- [Temporal Test-Time Adaptation with State-Space Models](https://mlsanity.com/paper/2407.12492/) (2024)
- [Dual-Phase Continual Learning: Supervised Adaptation Meets Unsupervised Retention](https://mlsanity.com/paper/2406.13653/) (2024)
- [Fully Test-Time rPPG Estimation via Synthetic Signal-Guided Feature Learning](https://mlsanity.com/paper/2407.13322/) (2024)
- [Towards Unified and Effective Domain Generalization](https://mlsanity.com/paper/2310.10008/) (2023)
- [Buffer layers for Test-Time Adaptation](https://mlsanity.com/paper/2510.21271/) (2025)
- [Resilient Practical Test-Time Adaptation: Soft Batch Normalization Alignment and Entropy-driven Memory Bank](https://mlsanity.com/paper/2401.14619/) (2024)
- [Is Less More? Exploring Token Condensation as Training-free Test-time Adaptation](https://mlsanity.com/paper/2410.14729/) (2024)
- [Towards Understanding Extrapolation: a Causal Lens](https://mlsanity.com/paper/2501.09163/) (2025)
- [Test-time Correlation Alignment](https://mlsanity.com/paper/2505.00533/) (2025)
- [Histopath-C: Towards Realistic Domain Shifts for Histopathology Vision-Language Adaptation](https://mlsanity.com/paper/2601.12493/) (2026)
- [Probabilistic Test-Time Generalization by Variational Neighbor-Labeling](https://mlsanity.com/paper/2307.04033/) (2023)
- [Generalized Robust Test-Time Adaptation in Continuous Dynamic Scenarios](https://mlsanity.com/paper/2310.04714/) (2023)
- [Test-Time Training for Semantic Segmentation with Output Contrastive Loss](https://mlsanity.com/paper/2311.07877/) (2023)
- [FunOTTA: On-the-Fly Adaptation on Cross-Domain Fundus Image via Stable Test-time Training](https://mlsanity.com/paper/2407.04396/) (2024)
- [Understanding and Mitigating Distribution Shifts For Machine Learning Force Fields](https://mlsanity.com/paper/2503.08674/) (2025)

## Abstract (arXiv)

Test-time adaptation (TTA) aims to adapt a trained classifier using online unlabeled test data only, without any information related to the training procedure. Most existing TTA methods adapt the trained classifier using the classifier's prediction on the test data as pseudo-label. However, under test-time domain shift, accuracy of the pseudo labels cannot be guaranteed, and thus the TTA methods often encounter performance degradation at the adapted classifier. To overcome this limitation, we propose a novel test-time adaptation method, called Test-time Adaptation via Self-Training with nearest neighbor information (TAST), which is composed of the following procedures: (1) adds trainable adaptation modules on top of the trained feature extractor; (2) newly defines a pseudo-label distribution for the test data by using the nearest neighbor information; (3) trains these modules only a few times during test time to match the nearest neighbor-based pseudo label distribution and a prototype-based class distribution for the test data; and (4) predicts the label of test data using the average predicted class distribution from these modules. The pseudo-label generation is based on the basic intuition that a test data and its nearest neighbor in the embedding space are likely to share the same label under the domain shift. By utilizing multiple randomly initialized adaptation modules, TAST extracts useful information for the classification of the test data under the domain shift, using the nearest neighbor information. TAST showed better performance than the state-of-the-art TTA methods on two standard benchmark tasks, domain generalization, namely VLCS, PACS, OfficeHome, and TerraIncognita, and image corruption, particularly CIFAR-10/100C.