# CAFA: Class-Aware Feature Alignment for Test-Time Adaptation

arXiv: https://arxiv.org/abs/2206.00205 · HTML: https://mlsanity.com/paper/2206.00205/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 80 references that are in the map.
- First posted on arXiv 1 Jun 2022.

## Builds on (most cited first, 30 of 46)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Moment Matching for Multi-Source Domain Adaptation](https://mlsanity.com/paper/1812.01754/) (2018)
- [Learning Transferable Features with Deep Adaptation Networks](https://mlsanity.com/paper/1502.02791/) (2015)
- [Adversarial Discriminative Domain Adaptation](https://mlsanity.com/paper/1702.05464/) (2017)
- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation](https://mlsanity.com/paper/1607.01719/) (2016)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://mlsanity.com/paper/1711.03213/) (2017)
- [Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2002.08546/) (2020)
- [Maximum Classifier Discrepancy for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1712.02560/) (2017)
- [Deep Transfer Learning with Joint Adaptation Networks](https://mlsanity.com/paper/1605.06636/) (2016)
- [Deep Domain Confusion: Maximizing for Domain Invariance](https://mlsanity.com/paper/1412.3474/) (2014)
- [Efficient Test-Time Model Adaptation without Forgetting](https://mlsanity.com/paper/2204.02610/) (2022)
- [Improving robustness against common corruptions by covariate shift adaptation](https://mlsanity.com/paper/2006.16971/) (2020)
- [MEMO: Test Time Robustness via Adaptation and Augmentation](https://mlsanity.com/paper/2110.09506/) (2021)
- [Domain Separation Networks](https://mlsanity.com/paper/1608.06019/) (2016)
- [Contrastive Test-Time Adaptation](https://mlsanity.com/paper/2204.10377/) (2022)
- [Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shift](https://mlsanity.com/paper/2006.10963/) (2020)
- [Universal Source-Free Domain Adaptation](https://mlsanity.com/paper/2004.04393/) (2020)
- [A DIRT-T Approach to Unsupervised Domain Adaptation](https://mlsanity.com/paper/1802.08735/) (2018)
- [RobustNet: Improving Domain Generalization in Urban-Scene Segmentation via Instance Selective Whitening](https://mlsanity.com/paper/2103.15597/) (2021)
- [Simultaneous Deep Transfer Across Domains and Tasks](https://mlsanity.com/paper/1510.02192/) (2015)

## Built on by (most cited first, 21 of 21)

- [A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts](https://mlsanity.com/paper/2303.15361/) (2023)
- [EcoTTA: Memory-Efficient Continual Test-time Adaptation via Self-distilled Regularization](https://mlsanity.com/paper/2303.01904/) (2023)
- [Towards Open-Set Test-Time Adaptation Utilizing the Wisdom of Crowds in Entropy Minimization](https://mlsanity.com/paper/2308.06879/) (2023)
- [BECoTTA: Input-dependent Online Blending of Experts for Continual Test-time Adaptation](https://mlsanity.com/paper/2402.08712/) (2024)
- [What, How, and When Should Object Detectors Update in Continually Changing Test Domains?](https://mlsanity.com/paper/2312.08875/) (2023)
- [CustomKD: Customizing Large Vision Foundation for Edge Model Improvement via Knowledge Distillation](https://mlsanity.com/paper/2503.18244/) (2025)
- [Monitoring Risks in Test-Time Adaptation](https://mlsanity.com/paper/2507.08721/) (2025)
- [Test-time Adaptation for Regression by Subspace Alignment](https://mlsanity.com/paper/2410.03263/) (2024)
- [Adaptive Concept Bottleneck for Foundation Models Under Distribution Shifts](https://mlsanity.com/paper/2412.14097/) (2024)
- [Variational Continual Test-Time Adaptation](https://mlsanity.com/paper/2402.08182/) (2024)
- [MOS: Model Synergy for Test-Time Adaptation on LiDAR-Based 3D Object Detection](https://mlsanity.com/paper/2406.14878/) (2024)
- [Adapting In-Domain Few-Shot Segmentation to New Domains without Source Domain Retraining](https://mlsanity.com/paper/2504.21414/) (2025)
- [Test-time Adaptation Meets Image Enhancement: Improving Accuracy via Uncertainty-aware Logit Switching](https://mlsanity.com/paper/2403.17423/) (2024)
- [Diverse Feature Learning by Self-distillation and Reset](https://mlsanity.com/paper/2403.19941/) (2024)
- [Test-Time Adaptation by Causal Trimming](https://mlsanity.com/paper/2510.11133/) (2025)
- [Enhanced Online Test-time Adaptation with Feature-Weight Cosine Alignment](https://mlsanity.com/paper/2405.07171/) (2024)
- [CodeMerge: Codebook-Guided Model Merging for Robust Test-Time Adaptation in Autonomous Driving](https://mlsanity.com/paper/2505.16524/) (2025)
- [Distill Once, Adapt Life-Long: Exploring Dataset Distillation for Continual Test-Time Adaptation](https://mlsanity.com/paper/2606.20196/) (2026)
- [Mitigating the Bias in the Model for Continual Test-Time Adaptation](https://mlsanity.com/paper/2403.01344/) (2024)
- [Stabilizing Test-Time Adaptation of High-Dimensional Simulation Surrogates via D-Optimal Statistics](https://mlsanity.com/paper/2602.15820/) (2026)
- [Can You Trust Frozen Hematology Foundation Models under Acquisition Shift?](https://mlsanity.com/paper/2608.25148/) (2026)

## Abstract (arXiv)

Despite recent advancements in deep learning, deep neural networks continue to suffer from performance degradation when applied to new data that differs from training data. Test-time adaptation (TTA) aims to address this challenge by adapting a model to unlabeled data at test time. TTA can be applied to pretrained networks without modifying their training procedures, enabling them to utilize a well-formed source distribution for adaptation. One possible approach is to align the representation space of test samples to the source distribution (textiti.e., feature alignment). However, performing feature alignment in TTA is especially challenging in that access to labeled source data is restricted during adaptation. That is, a model does not have a chance to learn test data in a class-discriminative manner, which was feasible in other adaptation tasks (textite.g., unsupervised domain adaptation) via supervised losses on the source data. Based on this observation, we propose a simple yet effective feature alignment loss, termed as Class-Aware Feature Alignment (CAFA), which simultaneously 1) encourages a model to learn target representations in a class-discriminative manner and 2) effectively mitigates the distribution shifts at test time. Our method does not require any hyper-parameters or additional losses, which are required in previous approaches. We conduct extensive experiments on 6 different datasets and show our proposed method consistently outperforms existing baselines.