# TeST: Test-time Self-Training under Distribution Shift

arXiv: https://arxiv.org/abs/2209.11459 · HTML: https://mlsanity.com/paper/2209.11459/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 54 of its 71 references that are in the map.
- First posted on arXiv 23 Sep 2022.

## Builds on (most cited first, 30 of 54)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [Adversarial Discriminative Domain Adaptation](https://mlsanity.com/paper/1702.05464/) (2017)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://mlsanity.com/paper/1711.03213/) (2017)
- [Deep Transfer Learning with Joint Adaptation Networks](https://mlsanity.com/paper/1605.06636/) (2016)
- [Deep Domain Confusion: Maximizing for Domain Invariance](https://mlsanity.com/paper/1412.3474/) (2014)
- [Born Again Neural Networks](https://mlsanity.com/paper/1805.04770/) (2018)
- [Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results](https://mlsanity.com/paper/1703.01780/) (2017)
- [Semantic Foggy Scene Understanding with Synthetic Data](https://mlsanity.com/paper/1708.07819/) (2017)
- [Semi-supervised Domain Adaptation via Minimax Entropy](https://mlsanity.com/paper/1904.06487/) (2019)
- [End-to-End Semi-Supervised Object Detection with Soft Teacher](https://mlsanity.com/paper/2106.09018/) (2021)
- [Learning to Generate Novel Domains for Domain Generalization](https://mlsanity.com/paper/2007.03304/) (2020)
- [Dilated Residual Networks](https://mlsanity.com/paper/1705.09914/) (2017)
- [Strong-Weak Distribution Alignment for Adaptive Object Detection](https://mlsanity.com/paper/1812.04798/) (2018)
- [Knowledge distillation: A good teacher is patient and consistent](https://mlsanity.com/paper/2106.05237/) (2021)

## Built on by (most cited first, 29 of 29)

- [A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts](https://mlsanity.com/paper/2303.15361/) (2023)
- [On Pitfalls of Test-Time Adaptation](https://mlsanity.com/paper/2306.03536/) (2023)
- [Each Test Image Deserves A Specific Prompt: Continual Test-Time Adaptation for 2D Medical Image Segmentation](https://mlsanity.com/paper/2311.18363/) (2023)
- [Revisiting Realistic Test-Time Training: Sequential Inference and Adaptation by Anchored Clustering Regularized Self-Training](https://mlsanity.com/paper/2303.10856/) (2023)
- [DARTH: Holistic Test-time Adaptation for Multiple Object Tracking](https://mlsanity.com/paper/2310.01926/) (2023)
- [What, How, and When Should Object Detectors Update in Continually Changing Test Domains?](https://mlsanity.com/paper/2312.08875/) (2023)
- [A Data-Centric Perspective on Evaluating Machine Learning Models for Tabular Data](https://mlsanity.com/paper/2407.02112/) (2024)
- [Uncovering Adversarial Risks of Test-Time Adaptation](https://mlsanity.com/paper/2301.12576/) (2023)
- [Gradient Alignment Improves Test-Time Adaptation for Medical Image Segmentation](https://mlsanity.com/paper/2408.07343/) (2024)
- [Parameter-Selective Continual Test-Time Adaptation](https://mlsanity.com/paper/2407.02253/) (2024)
- [TTT-KD: Test-Time Training for 3D Semantic Segmentation through Knowledge Distillation from Foundation Models](https://mlsanity.com/paper/2403.11691/) (2024)
- [Semi-Self-Supervised Domain Adaptation: Developing Deep Learning Models with Limited Annotated Data for Wheat Head Segmentation](https://mlsanity.com/paper/2405.07157/) (2024)
- [Test-time Correlation Alignment](https://mlsanity.com/paper/2505.00533/) (2025)
- [Source-Free Object Detection with Detection Transformer](https://mlsanity.com/paper/2510.11090/) (2025)
- [Multifaceted Evaluation of Audio-Visual Capability for MLLMs: Effectiveness, Efficiency, Generalizability and Robustness](https://mlsanity.com/paper/2504.16936/) (2025)
- [How Far Can LLMs Improve from Experience? Measuring Test-Time Learning Ability in LLMs with Human Comparison](https://mlsanity.com/paper/2506.14448/) (2025)
- [Test-time Adaptation Meets Image Enhancement: Improving Accuracy via Uncertainty-aware Logit Switching](https://mlsanity.com/paper/2403.17423/) (2024)
- [FRET: Feature Redundancy Elimination for Test Time Adaptation](https://mlsanity.com/paper/2505.10641/) (2025)
- [Few-Shot Test-Time Optimization Without Retraining for Semiconductor Recipe Generation and Beyond](https://mlsanity.com/paper/2505.16060/) (2025)
- [Stabilizing Open-Set Test-Time Adaptation via Primary-Auxiliary Filtering and Knowledge-Integrated Prediction](https://mlsanity.com/paper/2508.18751/) (2025)
- [Instance-Aware Test-Time Segmentation for Continual Domain Shifts](https://mlsanity.com/paper/2512.08569/) (2025)
- [CLAP: Contrastive Latent-space Prompt Optimization for End-to-end Autonomous Driving](https://mlsanity.com/paper/2605.17284/) (2026)
- [Test Time Training for Supervised Causal Learning](https://mlsanity.com/paper/2605.30015/) (2026)
- [Fairness-Aware Test-Time Prompt Tuning](https://mlsanity.com/paper/2608.25707/) (2026)
- [Open-World Test-Time Training: Self-Training with Contrast Learning](https://mlsanity.com/paper/2409.09591/) (2024)
- [Foundation Models for Medical Imaging: Status, Challenges, and Directions](https://mlsanity.com/paper/2602.15913/) (2026)
- [Predictive Spectral Calibration for Source-Free Test-Time Regression](https://mlsanity.com/paper/2603.09338/) (2026)
- [Sample-wise Targeted Adversarial Attacks on Test-time Adaptation](https://mlsanity.com/paper/2605.23411/) (2026)
- [Robust Online Aero-Engine Blade Defect Detection via Dual-Alignment Test-Time Adaptation](https://mlsanity.com/paper/2610.00067/) (2026)

## Abstract (arXiv)

Despite their recent success, deep neural networks continue to perform poorly when they encounter distribution shifts at test time. Many recently proposed approaches try to counter this by aligning the model to the new distribution prior to inference. With no labels available this requires unsupervised objectives to adapt the model on the observed test data. In this paper, we propose Test-Time Self-Training (TeST): a technique that takes as input a model trained on some source data and a novel data distribution at test time, and learns invariant and robust representations using a student-teacher framework. We find that models adapted using TeST significantly improve over baseline test-time adaptation algorithms. TeST achieves competitive performance to modern domain adaptation algorithms, while having access to 5-10x less data at time of adaption. We thoroughly evaluate a variety of baselines on two tasks: object detection and image segmentation and find that models adapted with TeST. We find that TeST sets the new state-of-the art for test-time domain adaptation algorithms.