# Diffusion-TTA: Test-time Adaptation of Discriminative Models via Generative Feedback

arXiv: https://arxiv.org/abs/2311.16102 · HTML: https://mlsanity.com/paper/2311.16102/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 39 of its 57 references that are in the map.
- First posted on arXiv 27 Nov 2023.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 30 of 39)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization](https://mlsanity.com/paper/2006.16241/) (2020)
- [Fine-Grained Visual Classification of Aircraft](https://mlsanity.com/paper/1306.5151/) (2013)
- [Do ImageNet Classifiers Generalize to ImageNet?](https://mlsanity.com/paper/1902.10811/) (2019)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [Context Encoders: Feature Learning by Inpainting](https://mlsanity.com/paper/1604.07379/) (2016)
- [Natural Adversarial Examples](https://mlsanity.com/paper/1907.07174/) (2019)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [GLIGEN: Open-Set Grounded Text-to-Image Generation](https://mlsanity.com/paper/2301.07093/) (2023)
- [Continual Test-Time Domain Adaptation](https://mlsanity.com/paper/2203.13591/) (2022)
- [Label-Efficient Semantic Segmentation with Diffusion Models](https://mlsanity.com/paper/2112.03126/) (2021)
- [Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models](https://mlsanity.com/paper/2209.07511/) (2022)
- [Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Models](https://mlsanity.com/paper/2303.04803/) (2023)
- [Adversarial Feature Learning](https://mlsanity.com/paper/1605.09782/) (2016)
- [Synthetic Data from Diffusion Models Improves ImageNet Classification](https://mlsanity.com/paper/2304.08466/) (2023)
- [Your Diffusion Model is Secretly a Zero-Shot Classifier](https://mlsanity.com/paper/2303.16203/) (2023)
- [Effective Data Augmentation With Diffusion Models](https://mlsanity.com/paper/2302.07944/) (2023)

## Built on by (most cited first, 27 of 27)

- [An Introduction to Vision-Language Modeling](https://mlsanity.com/paper/2405.17247/) (2024)
- [Unified Multimodal Discrete Diffusion](https://mlsanity.com/paper/2503.20853/) (2025)
- [Diffusion Feedback Helps CLIP See Better](https://mlsanity.com/paper/2407.20171/) (2024)
- [Advances in Multimodal Adaptation and Generalization: From Traditional Approaches to Foundation Models](https://mlsanity.com/paper/2501.18592/) (2025)
- [OT-VP: Optimal Transport-guided Visual Prompting for Test-Time Adaptation](https://mlsanity.com/paper/2407.09498/) (2024)
- [Noisy Test-Time Adaptation in Vision-Language Models](https://mlsanity.com/paper/2502.14604/) (2025)
- [Can World Models Benefit VLMs for World Dynamics?](https://mlsanity.com/paper/2510.00855/) (2025)
- [Generative Classifiers Avoid Shortcut Solutions](https://mlsanity.com/paper/2512.25034/) (2025)
- [Test-Time Adaptation of 3D Point Clouds via Denoising Diffusion Models](https://mlsanity.com/paper/2411.14495/) (2024)
- [Generative Learning for Forecasting the Dynamics of Complex Systems](https://mlsanity.com/paper/2402.17157/) (2024)
- [MOSAIC: Generating Consistent, Privacy-Preserving Scenes from Multiple Depth Views in Multi-Room Environments](https://mlsanity.com/paper/2503.13816/) (2025)
- [SketchFusion: Learning Universal Sketch Features through Fusing Foundation Models](https://mlsanity.com/paper/2503.14129/) (2025)
- [Test-Time Iterative Error Correction for Efficient Diffusion Models](https://mlsanity.com/paper/2511.06250/) (2025)
- [Asynchronous Perception Machine For Efficient Test-Time-Training](https://mlsanity.com/paper/2410.20535/) (2024)
- [Cross-modal Proxy Evolving for OOD Detection with Vision-Language Models](https://mlsanity.com/paper/2601.08476/) (2026)
- [GAIA: A Data Flywheel System for Training GUI Test-Time Scaling Critic Models](https://mlsanity.com/paper/2601.18197/) (2026)
- [Dual-frame Fluid Motion Estimation with Test-time Optimization and Zero-divergence Loss](https://mlsanity.com/paper/2410.11934/) (2024)
- [DIR: Retrieval-Augmented Image Captioning with Comprehensive Understanding](https://mlsanity.com/paper/2412.01115/) (2024)
- [Is Generation Required for Data-Efficient Perception?](https://mlsanity.com/paper/2512.08854/) (2025)
- [Mitigating the Bias in the Model for Continual Test-Time Adaptation](https://mlsanity.com/paper/2403.01344/) (2024)
- [Everything to the Synthetic: Diffusion-driven Test-time Adaptation via Synthetic-Domain Alignment](https://mlsanity.com/paper/2406.04295/) (2024)
- [SteeringTTA: Guiding Diffusion Trajectories for Robust Test-Time-Adaptation](https://mlsanity.com/paper/2510.14634/) (2025)
- [BRIC: Bridging Kinematic Plans and Physical Control at Test Time](https://mlsanity.com/paper/2511.20431/) (2025)
- [ProtoTTA: Prototype-Guided Test-Time Adaptation](https://mlsanity.com/paper/2604.15494/) (2026)
- [Noise2Map: End-to-End Diffusion Model for Semantic Segmentation and Change Detection](https://mlsanity.com/paper/2604.27889/) (2026)
- [Learning Topology-Aware Representations via Test-Time Adaptation for Anomaly Segmentation](https://mlsanity.com/paper/2606.28268/) (2026)
- [Position Aware Layer Queries for Test Time Training in Vision Language Models](https://mlsanity.com/paper/2609.34021/) (2026)

## Abstract (arXiv)

The advancements in generative modeling, particularly the advent of diffusion models, have sparked a fundamental question: how can these models be effectively used for discriminative tasks? In this work, we find that generative models can be great test-time adapters for discriminative models. Our method, Diffusion-TTA, adapts pre-trained discriminative models such as image classifiers, segmenters and depth predictors, to each unlabelled example in the test set using generative feedback from a diffusion model. We achieve this by modulating the conditioning of the diffusion model using the output of the discriminative model. We then maximize the image likelihood objective by backpropagating the gradients to discriminative model's parameters. We show Diffusion-TTA significantly enhances the accuracy of various large-scale pre-trained discriminative models, such as, ImageNet classifiers, CLIP models, image pixel labellers and image depth predictors. Diffusion-TTA outperforms existing test-time adaptation methods, including TTT-MAE and TENT, and particularly shines in online adaptation setups, where the discriminative model is continually adapted to each example in the test set. We provide access to code, results, and visualizations on our website: https://diffusion-tta.github.io/.