# Visual Anagrams: Generating Multi-View Optical Illusions with Diffusion Models

arXiv: https://arxiv.org/abs/2311.17919 · HTML: https://mlsanity.com/paper/2311.17919/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 24 of its 48 references that are in the map.
- First posted on arXiv 29 Nov 2023.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 24 of 24)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [Compositional Visual Generation with Composable Diffusion Models](https://mlsanity.com/paper/2206.01714/) (2022)
- [Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation](https://mlsanity.com/paper/2212.00774/) (2022)
- [Diffusion models as plug-and-play priors](https://mlsanity.com/paper/2206.09012/) (2022)
- [Reduce, Reuse, Recycle: Compositional Generation with Energy-Based Diffusion Models and MCMC](https://mlsanity.com/paper/2302.11552/) (2023)
- [Adversarial Examples that Fool both Computer Vision and Time-Limited Humans](https://mlsanity.com/paper/1802.08195/) (2018)
- [Learning to Compose Visual Relations](https://mlsanity.com/paper/2111.09297/) (2021)
- [Intriguing properties of generative classifiers](https://mlsanity.com/paper/2309.16779/) (2023)
- [Compositional Sculpting of Iterative Generative Processes](https://mlsanity.com/paper/2309.16115/) (2023)
- [Designing Perceptual Puzzles by Differentiating Probabilistic Programs](https://mlsanity.com/paper/2204.12301/) (2022)
- [Toward Quantifying Ambiguities in Artistic Images](https://mlsanity.com/paper/2008.09688/) (2020)
- [GANmouflage: 3D Object Nondetection with Texture Fields](https://mlsanity.com/paper/2201.07202/) (2022)

## Built on by (most cited first, 30 of 46)

- [Video-Guided Foley Sound Generation with Multimodal Controls](https://mlsanity.com/paper/2411.17698/) (2024)
- [StyleMaster: Stylize Your Video with Artistic Generation and Translation](https://mlsanity.com/paper/2412.07744/) (2024)
- [Rethinking Score Distillation as a Bridge Between Image Distributions](https://mlsanity.com/paper/2406.09417/) (2024)
- [Diffusion Illusions: Hiding Images in Plain Sight](https://mlsanity.com/paper/2312.03817/) (2023)
- [SyncTweedies: A General Generative Framework Based on Synchronized Diffusions](https://mlsanity.com/paper/2403.14370/) (2024)
- [Lossy Compression with Pretrained Diffusion Models](https://mlsanity.com/paper/2501.09815/) (2025)
- [Factorized Diffusion: Perceptual Illusions by Noise Decomposition](https://mlsanity.com/paper/2404.11615/) (2024)
- [Images that Sound: Composing Images and Sounds on a Single Canvas](https://mlsanity.com/paper/2405.12221/) (2024)
- [Diffusion-Based Visual Art Creation: A Survey and New Perspectives](https://mlsanity.com/paper/2408.12128/) (2024)
- [HiPrompt: Tuning-free Higher-Resolution Generation with Hierarchical MLLM Prompts](https://mlsanity.com/paper/2409.02919/) (2024)
- [Illusory VQA: Benchmarking and Enhancing Multimodal Models on Visual Illusions](https://mlsanity.com/paper/2412.08169/) (2024)
- [Anchor Token Matching: Implicit Structure Locking for Training-free AR Image Editing](https://mlsanity.com/paper/2504.10434/) (2025)
- [The Art of Deception: Color Visual Illusions and Diffusion Models](https://mlsanity.com/paper/2412.10122/) (2024)
- [StochSync: Stochastic Diffusion Synchronization for Image Generation in Arbitrary Spaces](https://mlsanity.com/paper/2501.15445/) (2025)
- [Unconditional Priors Matter! Improving Conditional Generation of Fine-Tuned Diffusion Models](https://mlsanity.com/paper/2503.20240/) (2025)
- [PTDiffusion: Free Lunch for Generating Optical Illusion Hidden Pictures with Phase-Transferred Diffusion Model](https://mlsanity.com/paper/2503.06186/) (2025)
- [Leveraging Semantic Attribute Binding for Free-Lunch Color Control in Diffusion Models](https://mlsanity.com/paper/2503.09864/) (2025)
- [LookingGlass: Generative Anamorphoses via Laplacian Pyramid Warping](https://mlsanity.com/paper/2504.08902/) (2025)
- [Diffusion-based Visual Anagram as Multi-task Learning](https://mlsanity.com/paper/2412.02693/) (2024)
- [MatchDiffusion: Training-free Generation of Match-cuts](https://mlsanity.com/paper/2411.18677/) (2024)
- [Illusion3D: 3D Multiview Illusion with 2D Diffusion Priors](https://mlsanity.com/paper/2412.09625/) (2024)
- [GPS as a Control Signal for Image Generation](https://mlsanity.com/paper/2501.12390/) (2025)
- [MOSAIC: Generating Consistent, Privacy-Preserving Scenes from Multiple Depth Views in Multi-Room Environments](https://mlsanity.com/paper/2503.13816/) (2025)
- [Hand-Shadow Poser](https://mlsanity.com/paper/2505.07012/) (2025)
- [SyncSDE: A Probabilistic Framework for Diffusion Synchronization](https://mlsanity.com/paper/2503.21555/) (2025)
- [Visual Anagrams Reveal Hidden Differences in Holistic Shape Processing Across Vision Models](https://mlsanity.com/paper/2507.00493/) (2025)
- [PickStyle: Video-to-Video Style Transfer with Context-Style Adapters](https://mlsanity.com/paper/2510.07546/) (2025)
- [Stroke of Surprise: Progressive Semantic Illusions in Vector Sketching](https://mlsanity.com/paper/2602.12280/) (2026)
- [FreSca: Scaling in Frequency Space Enhances Diffusion Models](https://mlsanity.com/paper/2504.02154/) (2025)
- [Refracting Reality: Generating Images with Realistic Transparent Objects](https://mlsanity.com/paper/2511.17340/) (2025)

## Abstract (arXiv)

We address the problem of synthesizing multi-view optical illusions: images that change appearance upon a transformation, such as a flip or rotation. We propose a simple, zero-shot method for obtaining these illusions from off-the-shelf text-to-image diffusion models. During the reverse diffusion process, we estimate the noise from different views of a noisy image, and then combine these noise estimates together and denoise the image. A theoretical analysis suggests that this method works precisely for views that can be written as orthogonal transformations, of which permutations are a subset. This leads to the idea of a visual anagram--an image that changes appearance under some rearrangement of pixels. This includes rotations and flips, but also more exotic pixel permutations such as a jigsaw rearrangement. Our approach also naturally extends to illusions with more than two views. We provide both qualitative and quantitative results demonstrating the effectiveness and flexibility of our method. Please see our project webpage for additional visualizations and results: https://dangeng.github.io/visual_anagrams/