# ZoomLDM: Latent Diffusion Model for multi-scale image generation

arXiv: https://arxiv.org/abs/2411.16969 · HTML: https://mlsanity.com/paper/2411.16969/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models.
- Builds on 40 of its 53 references that are in the map.
- First posted on arXiv 25 Nov 2024.

## Placed in

- Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/)

## Builds on (most cited first, 30 of 40)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [Image Super-Resolution via Iterative Refinement](https://mlsanity.com/paper/2104.07636/) (2021)
- [Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts](https://mlsanity.com/paper/2102.08981/) (2021)
- [Attention-based Deep Multiple Instance Learning](https://mlsanity.com/paper/1802.04712/) (2018)
- [The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale](https://mlsanity.com/paper/1811.00982/) (2018)
- [Exploiting Diffusion Prior for Real-World Image Super-Resolution](https://mlsanity.com/paper/2305.07015/) (2023)
- [Dual-stream Multiple Instance Learning Network for Whole Slide Image Classification with Self-supervised Contrastive Learning](https://mlsanity.com/paper/2011.08939/) (2020)
- [ResShift: Efficient Diffusion Model for Image Super-resolution by Residual Shifting](https://mlsanity.com/paper/2307.12348/) (2023)
- [Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learning](https://mlsanity.com/paper/2206.02647/) (2022)
- [PixArt-Sigma: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generation](https://mlsanity.com/paper/2403.04692/) (2024)
- [BACH: Grand Challenge on Breast Cancer Histology Images](https://mlsanity.com/paper/1808.04277/) (2018)
- [Diffusion Probabilistic Models beat GANs on Medical Images](https://mlsanity.com/paper/2212.07501/) (2022)
- [A Morphology Focused Diffusion Probabilistic Model for Synthesis of Histopathology Images](https://mlsanity.com/paper/2209.13167/) (2022)
- [Prompt-tuning latent diffusion models for inverse problems](https://mlsanity.com/paper/2310.01110/) (2023)
- [Efficient Diffusion Model for Image Restoration by Residual Shifting](https://mlsanity.com/paper/2403.07319/) (2024)
- [PathLDM: Text conditioned Latent Diffusion Model for Histopathology](https://mlsanity.com/paper/2309.00748/) (2023)

## Built on by (most cited first, 16 of 16)

- [The Prism Hypothesis: Harmonizing Semantic and Pixel Representations via Unified Autoencoding](https://mlsanity.com/paper/2512.19693/) (2025)
- [PixCell: A generative foundation model for digital histopathology images](https://mlsanity.com/paper/2506.05127/) (2025)
- [Chain-of-Zoom: Extreme Super-Resolution via Scale Autoregression and Preference Alignment](https://mlsanity.com/paper/2505.18600/) (2025)
- [TP-Blend: Textual-Prompt Attention Pairing for Precise Object-Style Blending in Diffusion Models](https://mlsanity.com/paper/2601.08011/) (2026)
- [Semantic-Guided Cross-Sensor Super Resolution of Remote Sensing Images: A Gated Dual Conditioning Flow Matching Model](https://mlsanity.com/paper/2510.23816/) (2025)
- [RSEdit: Text-Guided Image Editing for Remote Sensing](https://mlsanity.com/paper/2603.13708/) (2026)
- [Show the Signal, Hide the Noise: Spectral Forcing for Pixel-Space Diffusion](https://mlsanity.com/paper/2606.15236/) (2026)
- [CytoSyn: a Foundation Diffusion Model for Histopathology -- Tech Report](https://mlsanity.com/paper/2603.18089/) (2026)
- [Patch-to-Global: Random Patch Diffusion for Globally Consistent Megapixel Artifact Inpainting in Whole Slide Images](https://mlsanity.com/paper/2609.24116/) (2026)
- [Semi-Supervised Domain Adaptation with Latent Diffusion for Pathology Image Classification](https://mlsanity.com/paper/2601.17228/) (2026)
- [Structure-Guided Histopathology Synthesis via Dual-LoRA Diffusion](https://mlsanity.com/paper/2603.04565/) (2026)
- [Layout-Guided Controllable Pathology Image Generation with In-Context Diffusion Transformers](https://mlsanity.com/paper/2603.13386/) (2026)
- [STREAM: Stochastic Riemannian Flow Matching with Anisotropic Decoder for Digital Histopathology Image Generation](https://mlsanity.com/paper/2606.07036/) (2026)
- [Genesis: A Generative Engine for Hierarchical Satellite Image Synthesis](https://mlsanity.com/paper/2609.02683/) (2026)
- [Moore, Escher, Penrose: A Conformal Golden Braid](https://mlsanity.com/paper/2610.02210/) (2026)
- [BigCell: Generating Gigapixel Whole-Slide Images](https://mlsanity.com/paper/2610.14261/) (2026)

## Abstract (arXiv)

Diffusion models have revolutionized image generation, yet several challenges restrict their application to large-image domains, such as digital pathology and satellite imagery. Given that it is infeasible to directly train a model on 'whole' images from domains with potential gigapixel sizes, diffusion-based generative methods have focused on synthesizing small, fixed-size patches extracted from these images. However, generating small patches has limited applicability since patch-based models fail to capture the global structures and wider context of large images, which can be crucial for synthesizing (semantically) accurate samples. To overcome this limitation, we present ZoomLDM, a diffusion model tailored for generating images across multiple scales. Central to our approach is a novel magnification-aware conditioning mechanism that utilizes self-supervised learning (SSL) embeddings and allows the diffusion model to synthesize images at different 'zoom' levels, i.e., fixed-size patches extracted from large images at varying scales. ZoomLDM synthesizes coherent histopathology images that remain contextually accurate and detailed at different zoom levels, achieving state-of-the-art image generation quality across all scales and excelling in the data-scarce setting of generating thumbnails of entire large images. The multi-scale nature of ZoomLDM unlocks additional capabilities in large image generation, enabling computationally tractable and globally coherent image synthesis up to 4096 × 4096 pixels and 4× super-resolution. Additionally, multi-scale features extracted from ZoomLDM are highly effective in multiple instance learning experiments.