# Stop Throwing Away Discriminators! Re-using Adversaries for Test-Time Training

arXiv: https://arxiv.org/abs/2108.12280 · HTML: https://mlsanity.com/paper/2108.12280/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 38 references that are in the map.
- First posted on arXiv 26 Aug 2021.

## Builds on (most cited first, 29 of 29)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [When Does Label Smoothing Help?](https://mlsanity.com/paper/1906.02629/) (2019)
- [Adversarial Feature Learning](https://mlsanity.com/paper/1605.09782/) (2016)
- [Learning from Simulated and Unsupervised Images through Adversarial Training](https://mlsanity.com/paper/1612.07828/) (2016)
- [Embracing Imperfect Datasets: A Review of Deep Learning Solutions for Medical Image Segmentation](https://mlsanity.com/paper/1908.10454/) (2019)
- [Generative Adversarial Network in Medical Imaging: A Review](https://mlsanity.com/paper/1809.07294/) (2018)
- [Not-so-supervised: a survey of semi-supervised, multi-instance, and transfer learning in medical image analysis](https://mlsanity.com/paper/1804.06353/) (2018)
- [A Topological Loss Function for Deep-Learning based Image Segmentation using Persistent Homology](https://mlsanity.com/paper/1910.01877/) (2019)
- [Adversarially Learned Anomaly Detection](https://mlsanity.com/paper/1812.02288/) (2018)
- [Anatomically Constrained Neural Networks (ACNN): Application to Cardiac Image Enhancement and Segmentation](https://mlsanity.com/paper/1705.08302/) (2017)
- [Test-Time Adaptable Neural Networks for Robust Medical Image Segmentation](https://mlsanity.com/paper/2004.04668/) (2020)
- [A critical analysis of self-supervision, or what we can learn from a single image](https://mlsanity.com/paper/1904.13132/) (2019)
- [Amortised MAP Inference for Image Super-resolution](https://mlsanity.com/paper/1610.04490/) (2016)
- [Constrained-CNN losses for weakly supervised segmentation](https://mlsanity.com/paper/1805.04628/) (2018)
- [Anatomical Priors in Convolutional Networks for Unsupervised Biomedical Segmentation](https://mlsanity.com/paper/1903.03148/) (2019)
- [Post-DAE: Anatomically Plausible Segmentation via Post-Processing with Denoising Autoencoders](https://mlsanity.com/paper/2006.13791/) (2020)
- [Fence GAN: Towards Better Anomaly Detection](https://mlsanity.com/paper/1904.01209/) (2019)
- [Smoothness and Stability in GANs](https://mlsanity.com/paper/2002.04185/) (2020)
- [Unsupervised Deep Learning for Bayesian Brain MRI Segmentation](https://mlsanity.com/paper/1904.11319/) (2019)
- [On the Effectiveness of Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1712.06391/) (2017)
- [Cardiac MRI Segmentation with Strong Anatomical Guarantees](https://mlsanity.com/paper/1907.02865/) (2019)
- [Incorporating prior knowledge in medical image segmentation: a survey](https://mlsanity.com/paper/1607.01092/) (2016)
- [Memorization Precedes Generation: Learning Unsupervised GANs with Memory Networks](https://mlsanity.com/paper/1803.01500/) (2018)
- [Is Discriminator a Good Feature Extractor?](https://mlsanity.com/paper/1912.00789/) (2019)

## Built on by (most cited first, 4 of 4)

- [DLTTA: Dynamic Learning Rate for Test-time Adaptation on Cross-domain Medical Images](https://mlsanity.com/paper/2205.13723/) (2022)
- [vMFNet: Compositionality Meets Domain-generalised Segmentation](https://mlsanity.com/paper/2206.14538/) (2022)
- [Post-Deployment Adaptation with Access to Source Data via Federated Learning and Source-Target Remote Gradient Alignment](https://mlsanity.com/paper/2308.16735/) (2023)
- [Early Stopping Criteria for Training Generative Adversarial Networks in Biomedical Imaging](https://mlsanity.com/paper/2405.20987/) (2024)

## Abstract (arXiv)

Thanks to their ability to learn data distributions without requiring paired data, Generative Adversarial Networks (GANs) have become an integral part of many computer vision methods, including those developed for medical image segmentation. These methods jointly train a segmentor and an adversarial mask discriminator, which provides a data-driven shape prior. At inference, the discriminator is discarded, and only the segmentor is used to predict label maps on test images. But should we discard the discriminator? Here, we argue that the life cycle of adversarial discriminators should not end after training. On the contrary, training stable GANs produces powerful shape priors that we can use to correct segmentor mistakes at inference. To achieve this, we develop stable mask discriminators that do not overfit or catastrophically forget. At test time, we fine-tune the segmentor on each individual test instance until it satisfies the learned shape prior. Our method is simple to implement and increases model performance. Moreover, it opens new directions for re-using mask discriminators at inference. We release the code used for the experiments at https://vios-s.github.io/adversarial-test-time-training.