mlsanityData to 7 Oct 2026

Splicing ViT Features for Semantic Appearance Transfer

arXiv 2201.00424 ↗First posted 2 Jan 2022cs.CVcited by 209

Not on the map. Why?

Cited by 99 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present a method for semantically transferring the visual appearance of one natural image to another. Specifically, our goal is to generate an image in which objects in a source structure image are "painted" with the visual appearance of their semantically related objects in a target appearance image. Our method works by training a generator given only a single structure/appearance image pair as input. To integrate semantic information into our framework - a pivotal component in tackling this task - our key idea is to leverage a pre-trained and fixed Vision Transformer (ViT) model which serves as an external semantic prior. Specifically, we derive novel representations of structure and appearance extracted from deep ViT features, untwisting them from the learned self-attention modules. We then establish an objective function that splices the desired structure and appearance representations, interweaving them together in the space of ViT features. Our framework, which we term "Splice", does not involve adversarial training, nor does it require any additional input information such as semantic segmentation or correspondences, and can generate high-resolution results, e.g., work in HD. We demonstrate high quality results on a variety of in-the-wild image pairs, under significant variations in the number of objects, their pose and appearance.

Key facts. Cited by 209 papers in the map (data through 7 Oct 2026). Builds on 33 of its 42 references that are in the map. First posted on arXiv 2 Jan 2022.

Builds on

33 of 42 references.

References · 33

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2017Mask R-CNNcited by 7,259
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  7. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  8. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  9. 2017Deep Image Priorcited by 1,097
  10. 2019StarGAN v2: Diverse Image Synthesis for Multiple Domainscited by 1,095
  11. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,063
  12. 2017Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networkscited by 707
21 more
  1. 2020Contrastive Learning for Unpaired Image-to-Image Translationcited by 637
  2. 2017Unsupervised Image-to-Image Translation Networkscited by 602
  3. 2021Intriguing Properties of Vision Transformerscited by 446
  4. 2014Understanding Deep Image Representations by Inverting Themcited by 399
  5. 2021Deep ViT Features as Dense Visual Descriptorscited by 327
  6. 2017DualGAN: Unsupervised Dual Learning for Image-to-Image Translationcited by 310
  7. 2017Learning to Discover Cross-Domain Relations with Generative Adversarial Networkscited by 278
  8. 2020Swapping Autoencoder for Deep Image Manipulationcited by 215
  9. 2017Neural Style Transfer: A Reviewcited by 199
  10. 2019Style Transfer by Relaxed Optimal Transport and Self-Similaritycited by 167
  11. 2019Photorealistic Style Transfer via Wavelet Transformscited by 166
  12. 2018The Contextual Loss for Image Transformation with Non-Aligned Datacited by 135
  13. 2016Unsupervised Cross-Domain Image Generationcited by 125
  14. 2016Combining Markov Random Fields and Convolutional Neural Networks for Image Synthesiscited by 102
  15. 2016Controlling Perceptual Factors in Neural Style Transfercited by 99
  16. 2017Stable and Controllable Neural Texture Synthesis and Style Transfer Using Histogram Lossescited by 69
  17. 2020Deformable Style Transfercited by 31
  18. 2020TuiGAN: Learning Versatile Image-to-Image Translation with Two Unpaired Imagescited by 23
  19. 2016Semantic Style Transfer and Turning Two-Bit Doodles into Fine Artworkscited by 22
  20. 2020Structural-analogy from a Single Image Paircited by 14
  21. 2019Bidirectional One-Shot Unsupervised Domain Mappingcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 16; 2023: 58; 2024: 53; 2025: 51; 2026: 31.

209 papers.

Sort:

By area:

  1. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  2. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  3. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 897
  4. 2023Zero-shot Image-to-Image Translationcited by 503
  5. 2023Emergent Correspondence from Image Diffusioncited by 437
  6. 2023DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Datacited by 322
  7. 2022Text2LIVE: Text-Driven Layered Image and Video Editingcited by 312
  8. 2023Style Aligned Image Generation via Shared Attentioncited by 214
  9. 2023Encoder-based Domain Tuning for Fast Personalization of Text-to-Image Modelscited by 203
  10. 2022Neural Feature Fusion Fields: 3D Distillation of Self-Supervised 2D Image Representationscited by 195
  11. 2022SinNeRF: Training Neural Radiance Fields on Complex Scenes from a Single Imagecited by 165
  12. 2022Diffusion-based Image Translation using Disentangled Style and Content Representationcited by 148
38 more
  1. 2023Direct Inversion: Boosting Diffusion-based Editing with 3 Lines of Codecited by 145
  2. 2023Cross-Image Attention for Zero-Shot Appearance Transfercited by 135
  3. 2023Weakly Supervised 3D Open-vocabulary Segmentationcited by 133
  4. 2023Space-Time Diffusion Features for Zero-Shot Text-Driven Motion Transfercited by 124
  5. 2024Training-Free Consistent Text-to-Image Generationcited by 123
  6. 2022MagicPony: Learning Articulated 3D Animals in the Wildcited by 111
  7. 2024One-Step Image Translation with Text-to-Image Modelscited by 111
  8. 2023SINE: Semantic-driven Image-based NeRF Editing with Prior-guided Editing Fieldcited by 93
  9. 2023Diffusion Models for Imperceptible and Transferable Adversarial Attackcited by 93
  10. 2023FreeControl: Training-Free Spatial Control of Any Text-to-Image Diffusion Model with Any Conditioncited by 89
  11. 2024DreamScene360: Unconstrained Text-to-3D Scene Generation with Panoramic Gaussian Splattingcited by 81
  12. 2023Beyond First-Order Tweedie: Solving Inverse Problems using Latent Diffusioncited by 78
  13. 2023Dynamic Prompt Learning: Addressing Cross-Attention Leakage for Text-Based Image Editingcited by 76
  14. 2022NeRF-SOS: Any-View Self-supervised Object Segmentation on Complex Scenescited by 71
  15. 2023StyleDiffusion: Prompt-Embedding Inversion for Text-Based Editingcited by 68
  16. 2022LASSIE: Learning Articulated Shapes from Sparse Image Ensemble via 3D Part Discoverycited by 65
  17. 2024DINO-Tracker: Taming DINO for Self-Supervised Point Tracking in a Single Videocited by 64
  18. 2023Zero-Shot Contrastive Loss for Text-Guided Diffusion Image Style Transfercited by 59
  19. 2023ImageBrush: Learning Visual In-Context Instructions for Exemplar-Based Image Manipulationcited by 59
  20. 2024AGG: Amortized Generative 3D Gaussians for Single Image to 3Dcited by 51
  21. 2023Contrastive Denoising Score for Text-guided Latent Diffusion Image Editingcited by 49
  22. 2024Schedule Your Edit: A Simple yet Effective Diffusion Noise Schedule for Image Editingcited by 38
  23. 2025UniEdit-Flow: Unleashing Inversion and Editing in the Era of Flow Modelscited by 38
  24. 2022DeS3: Adaptive Attention-driven Self and Soft Shadow Removal using ViT Similaritycited by 37
  25. 2023ArtBank: Artistic Style Transfer with Pre-trained Diffusion Model and Implicit Style Prompt Bankcited by 36
  26. 2023SHAP-EDITOR: Instruction-guided Latent 3D Editing in Secondscited by 36
  27. 2023DiffFashion: Reference-based Fashion Design with Structure-aware Transfer by Diffusion Modelscited by 33
  28. 2024Ctrl-X: Controlling Structure and Appearance for Text-To-Image Generation Without Guidancecited by 33
  29. 2025Attention Distillation: A Unified Approach to Visual Characteristics Transfercited by 32
  30. 2024FlexiEdit: Frequency-Aware Latent Refinement for Enhanced Non-Rigid Editingcited by 31
  31. 2022Hi-LASSIE: High-Fidelity Articulated Shape and Skeleton Discovery from Sparse Image Ensemblecited by 30
  32. 2023Neural Congealing: Aligning Images to a Joint Semantic Atlascited by 30
  33. 2023Energy-Based Cross Attention for Bayesian Context Update in Text-to-Image Diffusion Modelscited by 30
  34. 2022Structure Representation Network and Uncertainty Feedback Learning for Dense Non-Uniform Fog Removalcited by 28
  35. 2024DreamMotion: Space-Time Self-Similar Score Distillation for Zero-Shot Video Editingcited by 28
  36. 2023MaskSketch: Unpaired Structure-guided Masked Image Generationcited by 27
  37. 2022360FusionNeRF: Panoramic Neural Radiance Fields with Joint Guidancecited by 23
  38. 2023MAG-Edit: Localized Image Editing in Complex Scenarios via Mask-Based Attention-Adjusted Guidancecited by 21

50 most cited of 209 All 209 (JSON)