mlsanityData to 14 Oct 2026

ZeST: Zero-Shot Material Transfer from a Single Image

arXiv 2404.06425 ↗First posted 9 Apr 2024cs.CVcited by 24

Not on the map. Why?

Abstract (arXiv)

We propose ZeST, a method for zero-shot material transfer to an object in the input image given a material exemplar image. ZeST leverages existing diffusion adapters to extract implicit material representation from the exemplar image. This representation is used to transfer the material using pre-trained inpainting diffusion model on the object in the input image using depth estimates as geometry cue and grayscale object shading as illumination cues. The method works on real images without any training resulting a zero-shot approach. Both qualitative and quantitative results on real and synthetic datasets demonstrate that ZeST outputs photorealistic images with transferred materials. We also show the application of ZeST to perform multiple edits and robust material assignment under different illuminations. Project Page: https://ttchengab.github.io/zest

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). Builds on 45 of its 51 references that are in the map. First posted on arXiv 9 Apr 2024.

Builds on

45 of 51 references.

References · 45

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  5. 2022Classifier-Free Diffusion Guidancecited by 5,847
  6. 2022Generative Adversarial Networkscited by 5,627
  7. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  8. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  9. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  10. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
  11. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 3,045
  12. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
33 more
  1. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
  2. 2021Vision Transformers for Dense Predictioncited by 1,803
  3. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
  4. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,464
  5. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,351
  6. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,163
  7. 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,106
  8. 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 781
  9. 2023MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editingcited by 727
  10. 2023MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generationcited by 536
  11. 2023Scaling up GANs for Text-to-Image Synthesiscited by 492
  12. 2023Uni-ControlNet: All-in-One Control to Text-to-Image Diffusion Modelscited by 359
  13. 2023Training-Free Layout Control with Cross-Attention Guidancecited by 313
  14. 2023TEXTure: Text-Guided Texturing of 3D Shapescited by 304
  15. 2023P+: Extended Textual Conditioning in Text-to-Image Generationcited by 242
  16. 2023Text2Tex: Text-driven Texture Synthesis via Diffusion Modelscited by 235
  17. 2023Subject-driven Text-to-Image Generation via Apprenticeship Learningcited by 212
  18. 2022ReCo: Region-Controlled Text-to-Image Generationcited by 179
  19. 2024InstanceDiffusion: Instance-level Control for Image Generationcited by 175
  20. 2023Expressive Text-to-Image Generation with Rich Textcited by 89
  21. 2023TexFusion: Synthesizing 3D Textures with Text-Guided Image Diffusion Modelscited by 87
  22. 2014Material Recognition in the Wild with the Materials in Context Databasecited by 79
  23. 2023LooseControl: Lifting ControlNet for Generalized Depth Conditioningcited by 68
  24. 2023OBJECT 3DIT: Language-guided 3D-aware Image Editingcited by 63
  25. 2023Diffusion Handles: Enabling 3D Edits for Diffusion Models by Lifting Activations to 3Dcited by 59
  26. 2024TextureDreamer: Image-guided Texture Synthesis through Geometry-aware Diffusioncited by 52
  27. 2019Flexible SVBRDF Capture with a Multi-Image Deep Networkcited by 50
  28. 2023Alchemist: Parametric Control of Material Properties with Diffusion Modelscited by 38
  29. 2023Material Palette: Extraction of Materials from a Single Imagecited by 28
  30. 2023Materialistic: Selecting Similar Materials in Imagescited by 26
  31. 2022A Dense Material Segmentation Dataset for Indoor and Outdoor Scene Parsingcited by 23
  32. 2024Learning Continuous 3D Words for Text-to-Image Generationcited by 16
  33. 2023In-the-wild Material Appearance Editing using Perceptual Attributescited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 12; 2026: 9.

24 papers.

Sort:

By area:

  1. 2025IntrinsicEdit: Precise generative image manipulation in intrinsic spacecited by 17
  2. 2024Conditional Balance: Improving Multi-Conditioning Trade-Offs in Image Generationcited by 9
  3. 2025MARBLE: Material Recomposition and Blending in CLIP-Spacecited by 9
  4. 2024Eye-for-an-eye: Appearance Transfer with Semantic Correspondence in Diffusion Modelscited by 8
  5. 2025MatSwap: Light-aware material transfers in imagescited by 6
  6. 2024LineArt: A Knowledge-guided Training-free High-quality Appearance Transfer for Design Drawing with Diffusion Modelcited by 5
  7. 2025EditCLIP: Representation Learning for Image Editingcited by 5
  8. 2025Controllable-Continuous Color Editing in Diffusion Model via Color Mappingcited by 4
  9. 2025MaRI: Material Retrieval Integration across Domainscited by 2
  10. 2025A Controllable Appearance Representation for Flexible Transfer and Editingcited by 2
  11. 2025PPS-Ctrl: Controllable Sim-to-Real Translation for Colonoscopy Depth Estimationcited by 2
  12. 2025RETRO: REthinking Tactile Representation Learning with Material PriOrscited by 1
12 more
  1. 2026TexEditor: Structure-Preserving Text-Driven Texture Editingcited by 1
  2. 2025Inverse Image-Based Rendering for Light Field Generation from Single Imagescited by 0
  3. 2025CoatFusion: Controllable Material Coating in Imagescited by 0
  4. 2025FROMAT: Multiview Material Appearance Transfer via Few-Shot Self-Attention Adaptationcited by 0
  5. 2026Alterbute: Editing Intrinsic Attributes of Objects in Imagescited by 0
  6. 2026FMMC: Harnessing the Power of Foundation Models for Accurate Material Classificationcited by 0
  7. 2026A training-free framework for high-fidelity appearance transfer via diffusion transformerscited by 0
  8. 2026DealMaTe: Multi-Dimensional Material Transfer via Diffusion Transformercited by 0
  9. 2026Controllable Texture Tiling with Transformed RoPE-Enhanced Diffusion Modelscited by 0
  10. 2026AISPO: Enhancing Depth Reliability for Robotic Manipulation of Non-Lambertian Objects via Affine-Invariant Shape Priorcited by 0
  11. 2026MatReplace: A Reference-Free, Conditioning-Aligned Benchmark for Material Replacement in Interior Scenescited by 0
  12. 2026LoGAN: Multilingual Font Localization with Generative Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025MaterialFusion: High-Quality, Zero-Shot, and Controllable Material Transfer with Diffusion Models
  2. 2026MaTe: Images Are All You Need for Material Transfer via Diffusion Transformer
  3. 2024ZeroComp: Zero-shot Object Compositing from Image Intrinsics via Diffusion
  4. 2024Material Anything: Generating Materials for Any 3D Object via Diffusion
  5. 2023ControlMat: A Controlled Generative Approach to Material Capture
  6. 2025MatMart: Material Reconstruction of 3D Objects via Diffusion
  7. 2026Texture Space Material Diffusion
  8. 2024Diffusion-based image inpainting with internal learning