# TreeDiffusion: Hierarchical Generative Clustering for Conditional Diffusion

arXiv: https://arxiv.org/abs/2410.16910 · HTML: https://mlsanity.com/paper/2410.16910/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 26 of its 40 references that are in the map.
- First posted on arXiv 22 Oct 2024.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Score-based Generative Modeling in Latent Space](https://mlsanity.com/paper/2106.05931/) (2021)
- [Diffusion Autoencoders: Toward a Meaningful and Decodable Representation](https://mlsanity.com/paper/2111.15640/) (2021)
- [Ladder Variational Autoencoders](https://mlsanity.com/paper/1602.02282/) (2016)
- [Variational Deep Embedding: An Unsupervised and Generative Approach to Clustering](https://mlsanity.com/paper/1611.05148/) (2016)
- [Variational Mixture-of-Experts Autoencoders for Multi-Modal Deep Generative Models](https://mlsanity.com/paper/1911.03393/) (2019)
- [KNN-Diffusion: Image Generation via Large-Scale Retrieval](https://mlsanity.com/paper/2204.02849/) (2022)
- [DiffuseVAE: Efficient, Controllable and High-Fidelity Generation from Low-Dimensional Latents](https://mlsanity.com/paper/2201.00308/) (2022)
- [Adaptive Neural Trees](https://mlsanity.com/paper/1807.06699/) (2018)
- [Breaking the Curse of Dimensionality: Diffusion Models Efficiently Learn Low-Dimensional Distributions](https://mlsanity.com/paper/2409.02426/) (2024)
- [InfoDiffusion: Representation Learning Using Information Maximizing Diffusion Models](https://mlsanity.com/paper/2306.08757/) (2023)
- [Self-Guided Diffusion Models](https://mlsanity.com/paper/2210.06462/) (2022)
- [Exploring the Limits of Deep Image Clustering using Pretrained Models](https://mlsanity.com/paper/2303.17896/) (2023)
- [Explicitly Minimizing the Blur Error of Variational Autoencoders](https://mlsanity.com/paper/2304.05939/) (2023)
- [Nonparametric Variational Auto-encoders for Hierarchical Representation Learning](https://mlsanity.com/paper/1703.07027/) (2017)
- [Tree Variational Autoencoders](https://mlsanity.com/paper/2306.08984/) (2023)
- [scTree: Discovering Cellular Hierarchies in the Presence of Batch Effects in scRNA-seq Data](https://mlsanity.com/paper/2406.19300/) (2024)

## Built on by (most cited first, 3 of 3)

- [Flow Matching with Optimized Subclass Priors for Medical Image Augmentation](https://mlsanity.com/paper/2605.16469/) (2026)
- [Wavelet Flow Matching for Time Series](https://mlsanity.com/paper/2609.39374/) (2026)
- [Pro-t-VAE: Closed-Form Student's t Mixtures for Robust Prototype Discovery](https://mlsanity.com/paper/2610.14894/) (2026)

## Abstract (arXiv)

Generative modeling and clustering are conventionally distinct tasks in machine learning. Variational Autoencoders (VAEs) have been widely explored for their ability to integrate both, providing a framework for generative clustering. However, while VAEs can learn meaningful cluster representations in latent space, they often struggle to generate high-quality samples. This paper addresses this problem by introducing TreeDiffusion, a deep generative model that conditions diffusion models on learned latent hierarchical cluster representations from a VAE to obtain high-quality, cluster-specific generations. Our approach consists of two steps: first, a VAE-based clustering model learns a hierarchical latent representation of the data. Second, a cluster-aware diffusion model generates realistic images conditioned on the learned hierarchical structure. We systematically compare the generative capabilities of our approach with those of alternative conditioning strategies. Empirically, we demonstrate that conditioning diffusion models on hierarchical cluster representations improves the generative performance on real-world datasets compared to other approaches. Moreover, a key strength of our method lies in its ability to generate images that are both representative and specific to each cluster, enabling more detailed visualization of the learned latent structure. Our approach addresses the generative limitations of VAE-based clustering approaches by leveraging their learned structure, thereby advancing the field of generative clustering.