# Illusion3D: 3D Multiview Illusion with 2D Diffusion Priors

arXiv: https://arxiv.org/abs/2412.09625 · HTML: https://mlsanity.com/paper/2412.09625/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models.
- Builds on 26 of its 61 references that are in the map.
- First posted on arXiv 12 Dec 2024.

## Placed in

- Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/)

## Builds on (most cited first, 26 of 26)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [Magic3D: High-Resolution Text-to-3D Content Creation](https://mlsanity.com/paper/2211.10440/) (2022)
- [ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation](https://mlsanity.com/paper/2305.16213/) (2023)
- [Accelerating 3D Deep Learning with PyTorch3D](https://mlsanity.com/paper/2007.08501/) (2020)
- [Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation](https://mlsanity.com/paper/2303.13873/) (2023)
- [Latent-NeRF for Shape-Guided Generation of 3D Shapes and Textures](https://mlsanity.com/paper/2211.07600/) (2022)
- [Patch Diffusion: Faster and More Data-Efficient Training of Diffusion Models](https://mlsanity.com/paper/2304.12526/) (2023)
- [Intriguing properties of generative classifiers](https://mlsanity.com/paper/2309.16779/) (2023)
- [Visual Anagrams: Generating Multi-View Optical Illusions with Diffusion Models](https://mlsanity.com/paper/2311.17919/) (2023)
- [Patched Denoising Diffusion Models For High-Resolution Image Synthesis](https://mlsanity.com/paper/2308.01316/) (2023)
- [Rethinking Score Distillation as a Bridge Between Image Distributions](https://mlsanity.com/paper/2406.09417/) (2024)
- [Diffusion Illusions: Hiding Images in Plain Sight](https://mlsanity.com/paper/2312.03817/) (2023)
- [Wired Perspectives: Multi-View Wire Art Embraces Generative AI](https://mlsanity.com/paper/2311.15421/) (2023)
- [CamDiff: Camouflage Image Augmentation via Diffusion Model](https://mlsanity.com/paper/2304.05469/) (2023)
- [Designing Perceptual Puzzles by Differentiating Probabilistic Programs](https://mlsanity.com/paper/2204.12301/) (2022)
- [Shadow Art Revisited: A Differentiable Rendering Based Approach](https://mlsanity.com/paper/2107.14539/) (2021)
- [Toward Quantifying Ambiguities in Artistic Images](https://mlsanity.com/paper/2008.09688/) (2020)
- [GANmouflage: 3D Object Nondetection with Texture Fields](https://mlsanity.com/paper/2201.07202/) (2022)
- [Constructing Printable Surfaces with View-Dependent Appearance](https://mlsanity.com/paper/2306.07449/) (2023)
- [Evolving Three Dimension (3D) Abstract Art: Fitting Concepts by Language](https://mlsanity.com/paper/2304.12932/) (2023)

## Built on by (most cited first, 5 of 5)

- [LookingGlass: Generative Anamorphoses via Laplacian Pyramid Warping](https://mlsanity.com/paper/2504.08902/) (2025)
- [Stroke of Surprise: Progressive Semantic Illusions in Vector Sketching](https://mlsanity.com/paper/2602.12280/) (2026)
- [ShadowDraw: From Any Object to Shadow-Drawing Compositional Art](https://mlsanity.com/paper/2512.05110/) (2025)
- [JanusMesh: Fast and Zero-Shot 3D Visual Illusion Generation via Cross-Space Denoising](https://mlsanity.com/paper/2606.20563/) (2026)
- [Mirror Illusion Art](https://mlsanity.com/paper/2607.02015/) (2026)

## Abstract (arXiv)

Automatically generating multiview illusions is a compelling challenge, where a single piece of visual content offers distinct interpretations from different viewing perspectives. Traditional methods, such as shadow art and wire art, create interesting 3D illusions but are limited to simple visual outputs (i.e., figure-ground or line drawing), restricting their artistic expressiveness and practical versatility. Recent diffusion-based illusion generation methods can generate more intricate designs but are confined to 2D images. In this work, we present a simple yet effective approach for creating 3D multiview illusions based on user-provided text prompts or images. Our method leverages a pre-trained text-to-image diffusion model to optimize the textures and geometry of neural 3D representations through differentiable rendering. When viewed from multiple angles, this produces different interpretations. We develop several techniques to improve the quality of the generated 3D multiview illusions. We demonstrate the effectiveness of our approach through extensive experiments and showcase illusion generation with diverse 3D forms.