mlsanityData to 7 Oct 2026

Diffusion Hyperfeatures: Searching Through Time and Space for Semantic Correspondence

arXiv 2305.14334 ↗First posted 23 May 2023cs.CVcited by 211

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Diffusion models have been shown to be capable of generating high-quality images, suggesting that they could contain meaningful internal representations. Unfortunately, the feature maps that encode a diffusion model's internal information are spread not only over layers of the network, but also over diffusion timesteps, making it challenging to extract useful descriptors. We propose Diffusion Hyperfeatures, a framework for consolidating multi-scale and multi-timestep feature maps into per-pixel feature descriptors that can be used for downstream tasks. These descriptors can be extracted for both synthetic and real images using the generation and inversion processes. We evaluate the utility of our Diffusion Hyperfeatures on the task of semantic keypoint correspondence: our method achieves superior performance on the SPair-71k real image benchmark. We also demonstrate that our method is flexible and transferable: our feature aggregation network trained on the inversion features of real image pairs can be used on the generation features of synthetic image pairs with unseen objects and compositions. Our code is available at https://diffusion-hyperfeatures.github.io.

Key facts. Cited by 211 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 37 of its 50 references that are in the map. First posted on arXiv 23 May 2023.

Builds on

37 of 50 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023ReVersion: Diffusion-Based Relation Inversion from Images23 Mar
  2. 2022Shape-Guided Diffusion with Inside-Outside Attention1 Dec

Other references · 35

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  7. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  8. 2020Denoising Diffusion Implicit Modelscited by 9,655
  9. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  10. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  11. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  12. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
23 more
  1. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,176
  2. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 942
  3. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 897
  4. 2021Label-Efficient Semantic Segmentation with Diffusion Modelscited by 495
  5. 2017Deep Layer Aggregationcited by 436
  6. 2021Deep ViT Features as Dense Visual Descriptorscited by 327
  7. 2022Unsupervised Semantic Segmentation by Distilling Feature Correspondencescited by 241
  8. 2017Photographic Image Synthesis with Cascaded Refinement Networkscited by 223
  9. 2019SPair-71k: A Large-scale Benchmark for Semantic Correspondencecited by 140
  10. 2014Hypercolumns for Object Segmentation and Fine-grained Localizationcited by 128
  11. 2020Articulation-aware Canonical Surface Mappingcited by 80
  12. 2021GAN-Supervised Dense Visual Alignmentcited by 61
  13. 2015Proposal Flowcited by 57
  14. 2022Zero-Shot Category-Level Object Pose Estimationcited by 57
  15. 2019Hyperpixel Flow: Semantic Correspondence with Multi-layer Neural Featurescited by 56
  16. 2020Learning to Compose Hypercolumns for Visual Correspondencecited by 53
  17. 2022CATs++: Boosting Cost Aggregation with Convolutions and Transformerscited by 53
  18. 2017Proposal Flow: Semantic Correspondences from Object Proposalscited by 46
  19. 2022Probabilistic Warp Consistency for Weakly-Supervised Semantic Correspondencescited by 30
  20. 2023Neural Congealing: Aligning Images to a Joint Semantic Atlascited by 30
  21. 2023ASIC: Aligning Sparse in-the-wild Image Collectionscited by 26
  22. 2017AnchorNet: A Weakly Supervised Network to Learn Geometry-sensitive Features For Semantic Matchingcited by 13
  23. 2019Is Discriminator a Good Feature Extractor?cited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 23; 2024: 65; 2025: 78; 2026: 45.

211 papers.

Sort:

By area:

  1. 2023TokenFlow: Consistent Diffusion Features for Consistent Video Editingcited by 440
  2. 2023A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondencecited by 297
  3. 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 285
  4. 2024Video Prediction Policy: A Generalist Robot Policy with Predictive Visual Representationscited by 279
  5. 2024Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillationcited by 276
  6. 2024Probing the 3D Awareness of Visual Foundation Modelscited by 175
  7. 2023Cross-Image Attention for Zero-Shot Appearance Transfercited by 135
  8. 2023State of the Art on Diffusion Models for Visual Computingcited by 126
  9. 2024Training-Free Consistent Text-to-Image Generationcited by 123
  10. 2023Unsupervised Semantic Correspondence Using Stable Diffusioncited by 120
  11. 2023Diffusion Model as Representation Learnercited by 95
  12. 2023Telling Left from Right: Identifying Geometry-Aware Semantic Correspondencecited by 85
38 more
  1. 2024Robo-ABC: Affordance Generalization Beyond Categories via Semantic Correspondence for Robot Manipulationcited by 69
  2. 2024SG-I2V: Self-Guided Trajectory Control in Image-to-Video Generationcited by 68
  3. 2024UltraPixel: Advancing Ultra-High-Resolution Image Synthesis to New Peakscited by 64
  4. 2024SpotlessSplats: Ignoring Distractors in 3D Gaussian Splattingcited by 61
  5. 2023On the Design Fundamentals of Diffusion Models: A Surveycited by 59
  6. 2023RealFill: Reference-Driven Generation for Authentic Image Completioncited by 58
  7. 2023Diffusion 3D Features (Diff3F): Decorating Untextured Shapes with Distilled Semantic Featurescited by 55
  8. 2024Diffusion Models and Representation Learning: A Surveycited by 53
  9. 2023Readout Guidance: Learning Control from Diffusion Featurescited by 52
  10. 2023Generative Models: What Do They Know? Do They Know Things? Let's Find Out!cited by 50
  11. 2023DiffCAD: Weakly-Supervised Probabilistic CAD Model Retrieval and Alignment from an RGB Imagecited by 48
  12. 2023Text-image Alignment for Diffusion-based Perceptioncited by 42
  13. 2024EmerDiff: Emerging Pixel-level Semantic Knowledge in Diffusion Modelscited by 42
  14. 2024CleanDIFT: Diffusion Features without Noisecited by 42
  15. 2024Track4Gen: Teaching Video Diffusion Models to Track Points Improves Video Generationcited by 42
  16. 2025Boosting Generative Image Modeling via Joint Image-Feature Synthesiscited by 42
  17. 2023SD4Match: Learning to Prompt Stable Diffusion Model for Semantic Matchingcited by 38
  18. 2024Looking Backward: Streaming Video-to-Video Translation with Feature Bankscited by 38
  19. 2024Disentangled 3D Scene Generation with Layout Learningcited by 33
  20. 2023A General Protocol to Probe Large Vision Models for 3D Physical Understandingcited by 32
  21. 2025Emergent Temporal Correspondences from Video Diffusion Transformerscited by 30
  22. 2023FreeReg: Image-to-Point Cloud Registration Leveraging Pretrained Diffusion Models and Monocular Depth Estimatorscited by 29
  23. 2024Open-Vocabulary Attention Maps with Token Optimization for Semantic Segmentation in Diffusion Modelscited by 29
  24. 2024textitRevelio: Interpreting and leveraging semantic information in diffusion modelscited by 29
  25. 2025Diff2Flow: Training Flow Matching Models via Diffusion Model Alignmentcited by 27
  26. 2024DiffCut: Catalyzing Zero-Shot Semantic Segmentation with Diffusion Features and Recursive Normalized Cutcited by 26
  27. 2026Diamond Maps: Efficient Reward Alignment via Stochastic Flow Mapscited by 26
  28. 2024Not All Diffusion Model Activations Have Been Evaluated as Discriminative Featurescited by 23
  29. 2024AV-Link: Temporally-Aligned Diffusion Features for Cross-Modal Audio-Video Generationcited by 21
  30. 2023CLiC: Concept Learning in Contextcited by 20
  31. 2025Generative Uncertainty in Diffusion Modelscited by 20
  32. 2024When Does Perceptual Alignment Benefit Vision Representations?cited by 18
  33. 2025Understanding Representation Dynamics of Diffusion Models via Low-Dimensional Modelingcited by 18
  34. 2024Autogenic Language Embedding for Coherent Point Trackingcited by 17
  35. 2024Distillation of Diffusion Features for Semantic Correspondencecited by 17
  36. 2024DreamMover: Leveraging the Prior of Diffusion Models for Image Interpolation with Large Motioncited by 16
  37. 2024Unleashing the Potential of the Diffusion Model in Few-shot Semantic Segmentationcited by 16
  38. 2023Unsupervised Keypoints from Pretrained Diffusion Modelscited by 15

50 most cited of 211 All 211 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Emergent Correspondence from Image Diffusion
  2. 2025Unleashing Diffusion Transformers for Visual Correspondence by Modulating Massive Activations
  3. 2024Diffusion Models in Low-Level Vision: A Survey
  4. 2023DiffusionSeg: Adapting Diffusion Towards Unsupervised Object Discovery
  5. 2023Hierarchical Diffusion Autoencoders and Disentangled Image Manipulation
  6. 2024Diffusion Models Need Visual Priors for Image Generation
  7. 2024Dual Diffusion for Unified Image Generation and Understanding
  8. 2024Diffusion-RWKV: Scaling RWKV-Like Architectures for Diffusion Models