mlsanityData to 7 Oct 2026

Exploring Structured Semantic Priors Underlying Diffusion Score for Test-time Adaptation

arXiv 2501.00873 ↗First posted 1 Jan 2025cs.CV cs.LGcited by 2

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Capitalizing on the complementary advantages of generative and discriminative models has always been a compelling vision in machine learning, backed by a growing body of research. This work discloses the hidden semantic structure within score-based generative models, unveiling their potential as effective discriminative priors. Inspired by our theoretical findings, we propose DUSA to exploit the structured semantic priors underlying diffusion score to facilitate the test-time adaptation of image classifiers or dense predictors. Notably, DUSA extracts knowledge from a single timestep of denoising diffusion, lifting the curse of Monte Carlo-based likelihood estimation over timesteps. We demonstrate the efficacy of our DUSA in adapting a wide variety of competitive pre-trained discriminative models on diverse test-time scenarios. Additionally, a thorough ablation study is conducted to dissect the pivotal elements in DUSA. Code is publicly available at https://github.com/BIT-DA/DUSA.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 80 of its 104 references that are in the map. First posted on arXiv 1 Jan 2025.

Builds on

80 of 104 references.

Closest ancestors · 15

Same direction, newest first

  1. 2024Target Score Matching13 Feb
  2. 2023NoiseCLR: A Contrastive Learning Approach for Unsupervised Discovery of Interpretable Directions in Diffusion Models8 Dec
  3. 2023Do text-free diffusion models learn discriminative visual representations?29 Nov
  4. 2023Understanding the Latent Space of Diffusion Models through the Lens of Riemannian Geometry24 Jul
  5. 2023Discovering Interpretable Directions in the Semantic Latent Space of Diffusion Models20 Mar
  6. 2022Scalable Diffusion Models with Transformers19 Dec
  7. 2022Diffusion Models already have a Semantic Latent Space20 Oct
  8. 2022Understanding Diffusion Models: A Unified Perspective25 Aug
  9. 2022Your ViT is Secretly a Hybrid Discriminative-Generative Diffusion Model16 Aug
  10. 2022Classifier-Free Diffusion Guidance26 Jul
  11. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  12. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models6 Aug
3 more
  1. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  2. 2020Denoising Diffusion Probabilistic Models19 Jun
  3. 2020Improved Techniques for Training Score-Based Generative Models16 Jun

Other references · 65

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  6. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  7. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  8. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  9. 2022A ConvNet for the 2020scited by 3,545
  10. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
  11. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  12. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
38 more
  1. 2020Shortcut Learning in Deep Neural Networkscited by 1,928
  2. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  3. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
  4. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,168
  5. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
  6. 2016InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Netscited by 858
  7. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Modelscited by 696
  8. 2018Recognition in Terra Incognitacited by 655
  9. 2022Continual Test-Time Domain Adaptationcited by 520
  10. 2021Label-Efficient Semantic Segmentation with Diffusion Modelscited by 495
  11. 2022Diffusion Models for Adversarial Purificationcited by 469
  12. 2022Efficient Test-Time Model Adaptation without Forgettingcited by 467
  13. 2020Improving robustness against common corruptions by covariate shift adaptationcited by 427
  14. 2023Towards Stable Test-Time Adaptation in Dynamic Wild Worldcited by 409
  15. 2019Your Classifier is Secretly an Energy Based Model and You Should Treat it Like Onecited by 402
  16. 2021MEMO: Test Time Robustness via Adaptation and Augmentationcited by 376
  17. 2023Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Modelscited by 371
  18. 2016Adversarial Feature Learningcited by 350
  19. 2019Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Comingcited by 332
  20. 2023Your Diffusion Model is Secretly a Zero-Shot Classifiercited by 301
  21. 2023A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondencecited by 297
  22. 2016Revisiting Batch Normalization For Practical Domain Adaptationcited by 285
  23. 2021How to Train Your Energy-Based Modelscited by 271
  24. 2023Unleashing Text-to-Image Diffusion Models for Visual Perceptioncited by 266
  25. 2022Mitigating Neural Network Overconfidence with Logit Normalizationcited by 251
  26. 2020Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shiftcited by 235
  27. 2022MAGE: MAsked Generative Encoder to Unify Representation Learning and Image Synthesiscited by 222
  28. 2022Test-Time Training with Masked Autoencoderscited by 208
  29. 2023Robust Test-Time Adaptation in Dynamic Scenarioscited by 185
  30. 2022Parameter-free Online Test-time Adaptationcited by 184
  31. 2022Sketch-Guided Text-to-Image Diffusion Modelscited by 180
  32. 2023Text-to-Image Diffusion Models are Zero-Shot Classifierscited by 140
  33. 2023Unsupervised Semantic Correspondence Using Stable Diffusioncited by 120
  34. 2023Return of Unconditional Generation: A Self-supervised Representation Generation Methodcited by 95
  35. 2022Back to the Source: Diffusion-Driven Test-Time Adaptationcited by 71
  36. 2023The effectiveness of MAE pre-pretraining for billion-scale pretrainingcited by 68
  37. 2021MT3: Meta Test-Time Training for Self-Supervised Test-Time Adaptioncited by 65
  38. 2023Guided Image Synthesis via Initial Image Editing in Diffusion Modelcited by 63

50 most cited of 65 All 65 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Unlocking Diffusion Hierarchies: Adaptive Timestep Selection for Zero-Shot Segmentationcited by 1
  2. 2026Test-Time Adaptation in Optical Coherence Tomography Using Trajectory-Aligned Time-Independent Flowcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Diffusion-TTA: Test-time Adaptation of Discriminative Models via Generative Feedback
  2. 2023Turn Down the Noise: Leveraging Diffusion Models for Test-time Adaptation via Pseudo-label Ensembling
  3. 2023Two Simple Principles for Diffusion-Based Test-Time Adaptation
  4. 2025Guiding Noisy Label Conditional Diffusion Models with Score-based Discriminator Correction
  5. 2025Diffusion Models as Dataset Distillation Priors
  6. 2025Test-Time Modification: Inverse Domain Transformation for Robust Perception
  7. 2025Low-dimensional adaptation of diffusion models: Convergence in total variation
  8. 2026Probing Diffusion Denoising Dynamics for Contrastive Representation Learning