# ActMAD: Activation Matching to Align Distributions for Test-Time-Training

arXiv: https://arxiv.org/abs/2211.12870 · HTML: https://mlsanity.com/paper/2211.12870/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 50 references that are in the map.
- First posted on arXiv 23 Nov 2022.

## Builds on (most cited first, 30 of 39)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization](https://mlsanity.com/paper/2006.16241/) (2020)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour](https://mlsanity.com/paper/1706.02677/) (2017)
- [Learning Transferable Features with Deep Adaptation Networks](https://mlsanity.com/paper/1502.02791/) (2015)
- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation](https://mlsanity.com/paper/1607.01719/) (2016)
- [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016)
- [AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty](https://mlsanity.com/paper/1912.02781/) (2019)
- [Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2002.08546/) (2020)
- [Continual Test-Time Domain Adaptation](https://mlsanity.com/paper/2203.13591/) (2022)
- [Efficient Test-Time Model Adaptation without Forgetting](https://mlsanity.com/paper/2204.02610/) (2022)
- [Improving robustness against common corruptions by covariate shift adaptation](https://mlsanity.com/paper/2006.16971/) (2020)
- [Semantic Foggy Scene Understanding with Synthetic Data](https://mlsanity.com/paper/1708.07819/) (2017)
- [First return, then explore](https://mlsanity.com/paper/2004.12919/) (2020)
- [Test-Time Training with Masked Autoencoders](https://mlsanity.com/paper/2209.07522/) (2022)
- [Central Moment Discrepancy (CMD) for Domain-Invariant Representation Learning](https://mlsanity.com/paper/1702.08811/) (2017)
- [Parameter-free Online Test-time Adaptation](https://mlsanity.com/paper/2201.05718/) (2022)

## Built on by (most cited first, 30 of 41)

- [A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts](https://mlsanity.com/paper/2303.15361/) (2023)
- [Align Your Prompts: Test-Time Prompting with Distribution Alignment for Zero-Shot Generalization](https://mlsanity.com/paper/2311.01459/) (2023)
- [Test-Time Model Adaptation with Only Forward Passes](https://mlsanity.com/paper/2404.01650/) (2024)
- [Test-Time Learning for Large Language Models](https://mlsanity.com/paper/2505.20633/) (2025)
- [LaFTer: Label-Free Tuning of Zero-shot Classifier using Language and Unlabeled Image Collections](https://mlsanity.com/paper/2305.18287/) (2023)
- [DPCore: Dynamic Prompt Coreset for Continual Test-Time Adaptation](https://mlsanity.com/paper/2406.10737/) (2024)
- [PointSAM: Pointly-Supervised Segment Anything Model for Remote Sensing Images](https://mlsanity.com/paper/2409.13401/) (2024)
- [OT-VP: Optimal Transport-guided Visual Prompting for Test-Time Adaptation](https://mlsanity.com/paper/2407.09498/) (2024)
- [Evaluation of Test-Time Adaptation Under Computational Time Constraints](https://mlsanity.com/paper/2304.04795/) (2023)
- [What, How, and When Should Object Detectors Update in Continually Changing Test Domains?](https://mlsanity.com/paper/2312.08875/) (2023)
- [Test-Time Model Adaptation for Quantized Neural Networks](https://mlsanity.com/paper/2508.02180/) (2025)
- [TTRV: Test-Time Reinforcement Learning for Vision Language Models](https://mlsanity.com/paper/2510.06783/) (2025)
- [Online Continual Learning for Robust Indoor Object Recognition](https://mlsanity.com/paper/2307.09827/) (2023)
- [Self-Bootstrapping for Versatile Test-Time Adaptation](https://mlsanity.com/paper/2504.08010/) (2025)
- [IT3: Idempotent Test-Time Training](https://mlsanity.com/paper/2410.04201/) (2024)
- [TTT-KD: Test-Time Training for 3D Semantic Segmentation through Knowledge Distillation from Foundation Models](https://mlsanity.com/paper/2403.11691/) (2024)
- [Synchronizing Task Behavior: Aligning Multiple Tasks during Test-Time Training](https://mlsanity.com/paper/2507.07778/) (2025)
- [Towards Understanding Extrapolation: a Causal Lens](https://mlsanity.com/paper/2501.09163/) (2025)
- [Weakly Supervised Test-Time Domain Adaptation for Object Detection](https://mlsanity.com/paper/2407.05607/) (2024)
- [Style-Adaptive Detection Transformer for Single-Source Domain Generalized Object Detection](https://mlsanity.com/paper/2504.20498/) (2025)
- [Class-aware Domain Knowledge Fusion and Fission for Continual Test-Time Adaptation](https://mlsanity.com/paper/2510.12150/) (2025)
- [Test-Time Spectrum-Aware Latent Steering for Zero-Shot Generalization in Vision-Language Models](https://mlsanity.com/paper/2511.09809/) (2025)
- [FOZO: Forward-Only Zeroth-Order Prompt Optimization for Test-Time Adaptation](https://mlsanity.com/paper/2603.04733/) (2026)
- [Structural Alignment Improves Graph Test-Time Adaptation](https://mlsanity.com/paper/2502.18334/) (2025)
- [PAID: Pairwise Angular-Invariant Decomposition for Continual Test-Time Adaptation](https://mlsanity.com/paper/2506.02453/) (2025)
- [Sit Back and Relax: Learning to Drive Incrementally in All Weather Conditions](https://mlsanity.com/paper/2305.18953/) (2023)
- [Enhanced Online Test-time Adaptation with Feature-Weight Cosine Alignment](https://mlsanity.com/paper/2405.07171/) (2024)
- [Event Stream-based Visual Object Tracking: HDETrack V2 and A High-Definition Benchmark](https://mlsanity.com/paper/2502.05574/) (2025)
- [Continual Adaptation: Environment-Conditional Parameter Generation for Object Detection in Dynamic Scenarios](https://mlsanity.com/paper/2506.24063/) (2025)
- [Mitigating the Bias in the Model for Continual Test-Time Adaptation](https://mlsanity.com/paper/2403.01344/) (2024)

## Abstract (arXiv)

Test-Time-Training (TTT) is an approach to cope with out-of-distribution (OOD) data by adapting a trained model to distribution shifts occurring at test-time. We propose to perform this adaptation via Activation Matching (ActMAD): We analyze activations of the model and align activation statistics of the OOD test data to those of the training data. In contrast to existing methods, which model the distribution of entire channels in the ultimate layer of the feature extractor, we model the distribution of each feature in multiple layers across the network. This results in a more fine-grained supervision and makes ActMAD attain state of the art performance on CIFAR-100C and Imagenet-C. ActMAD is also architecture- and task-agnostic, which lets us go beyond image classification, and score 15.4% improvement over previous approaches when evaluating a KITTI-trained object detector on KITTI-Fog. Our experiments highlight that ActMAD can be applied to online adaptation in realistic scenarios, requiring little data to attain its full performance.