# MaterialFusion: High-Quality, Zero-Shot, and Controllable Material Transfer with Diffusion Models

arXiv: https://arxiv.org/abs/2502.06606 · HTML: https://mlsanity.com/paper/2502.06606/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models.
- Builds on 20 of its 25 references that are in the map.
- First posted on arXiv 10 Feb 2025.

## Placed in

- Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/)

## Builds on (most cited first, 20 of 20)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2308.06721/) (2023)
- [Null-text Inversion for Editing Real Images using Guided Diffusion Models](https://mlsanity.com/paper/2211.09794/) (2022)
- [Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translation](https://mlsanity.com/paper/2211.12572/) (2022)
- [Diffusion Self-Guidance for Controllable Image Generation](https://mlsanity.com/paper/2306.00986/) (2023)
- [TEXTure: Text-Guided Texturing of 3D Shapes](https://mlsanity.com/paper/2302.01721/) (2023)
- [Text2Tex: Text-driven Texture Synthesis via Diffusion Models](https://mlsanity.com/paper/2303.11396/) (2023)
- [TextureDreamer: Image-guided Texture Synthesis through Geometry-aware Diffusion](https://mlsanity.com/paper/2401.09416/) (2024)
- [Alchemist: Parametric Control of Material Properties with Diffusion Models](https://mlsanity.com/paper/2312.02970/) (2023)
- [Guide-and-Rescale: Self-Guidance Mechanism for Effective Tuning-Free Real Image Editing](https://mlsanity.com/paper/2409.01322/) (2024)
- [In-the-wild Material Appearance Editing using Perceptual Attributes](https://mlsanity.com/paper/2302.03619/) (2023)

## Built on by (most cited first, 10 of 10)

- [MatSwap: Light-aware material transfers in images](https://mlsanity.com/paper/2502.07784/) (2025)
- [Controllable-Continuous Color Editing in Diffusion Model via Color Mapping](https://mlsanity.com/paper/2509.13756/) (2025)
- [Overpainting: Localized Context-aware Diffusion Image Editing](https://mlsanity.com/paper/2609.10811/) (2026)
- [Recolour What Matters: Region-Aware Colour Editing via Token-Level Diffusion](https://mlsanity.com/paper/2603.18466/) (2026)
- [CoatFusion: Controllable Material Coating in Images](https://mlsanity.com/paper/2512.02143/) (2025)
- [FROMAT: Multiview Material Appearance Transfer via Few-Shot Self-Attention Adaptation](https://mlsanity.com/paper/2512.09617/) (2025)
- [Alterbute: Editing Intrinsic Attributes of Objects in Images](https://mlsanity.com/paper/2601.10714/) (2026)
- [A training-free framework for high-fidelity appearance transfer via diffusion transformers](https://mlsanity.com/paper/2603.26767/) (2026)
- [MaTe: Images Are All You Need for Material Transfer via Diffusion Transformer](https://mlsanity.com/paper/2605.15660/) (2026)
- [DealMaTe: Multi-Dimensional Material Transfer via Diffusion Transformer](https://mlsanity.com/paper/2605.15681/) (2026)

## Abstract (arXiv)

Manipulating the material appearance of objects in images is critical for applications like augmented reality, virtual prototyping, and digital content creation. We present MaterialFusion, a novel framework for high-quality material transfer that allows users to adjust the degree of material application, achieving an optimal balance between new material properties and the object's original features. MaterialFusion seamlessly integrates the modified object into the scene by maintaining background consistency and mitigating boundary artifacts. To thoroughly evaluate our approach, we have compiled a dataset of real-world material transfer examples and conducted complex comparative analyses. Through comprehensive quantitative evaluations and user studies, we demonstrate that MaterialFusion significantly outperforms existing methods in terms of quality, user control, and background preservation. Code is available at https://github.com/ControlGenAI/MaterialFusion.