# Category-based Galaxy Image Generation via Diffusion Models

arXiv: https://arxiv.org/abs/2506.16255 · HTML: https://mlsanity.com/paper/2506.16255/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models.
- Builds on 9 of its 81 references that are in the map.
- First posted on arXiv 19 Jun 2025.

## Placed in

- Diffusion models / [Inverse problems and image restoration](https://mlsanity.com/diffusion/inverse-problems-and-image-restoration/)

## Builds on (most cited first, 9 of 9)

- [Rotation-invariant convolutional neural networks for galaxy morphology prediction](https://mlsanity.com/paper/1503.07077/) (2015)
- [Galaxy Zoo DECaLS: Detailed Visual Morphology Measurements from Volunteers and Deep Learning for 314,000 Galaxies](https://mlsanity.com/paper/2102.08414/) (2021)
- [Realistic galaxy image simulation via score-based generative models](https://mlsanity.com/paper/2111.01713/) (2021)
- [Galaxy Zoo: Probabilistic Morphology through Bayesian CNNs and Active Learning](https://mlsanity.com/paper/1905.07424/) (2019)
- [Optimal Ratio for Data Splitting](https://mlsanity.com/paper/2202.03326/) (2022)
- [DeepMerge II: Building Robust Deep Learning Algorithms for Merging Galaxy Identification Across Domains](https://mlsanity.com/paper/2103.01373/) (2021)
- [Stokes Inversion based on Convolutional Neural Networks](https://mlsanity.com/paper/1904.03714/) (2019)
- [Forging new worlds: high-resolution synthetic galaxies with chained generative adversarial networks](https://mlsanity.com/paper/1811.03081/) (2018)
- [From Images to Features: Unbiased Morphology Classification via Variational Auto-Encoders and Domain Adaptation](https://mlsanity.com/paper/2303.08627/) (2023)

## Built on by (most cited first, 1 of 1)

- [Multi-Modal Masked Autoencoders for Learning Image-Spectrum Associations for Galaxy Evolution and Cosmology](https://mlsanity.com/paper/2510.22527/) (2025)

## Abstract (arXiv)

Conventional galaxy generation methods rely on semi-analytical models and hydrodynamic simulations, which are highly dependent on physical assumptions and parameter tuning. In contrast, data-driven generative models do not have explicit physical parameters pre-determined, and instead learn them efficiently from observational data, making them alternative solutions to galaxy generation. Among these, diffusion models outperform Variational Autoencoders (VAEs) and Generative Adversarial Networks (GANs) in quality and diversity. Leveraging physical prior knowledge to these models can further enhance their capabilities. In this work, we present GalCatDiff, the first framework in astronomy to leverage both galaxy image features and astrophysical properties in the network design of diffusion models. GalCatDiff incorporates an enhanced U-Net and a novel block entitled Astro-RAB (Residual Attention Block), which dynamically combines attention mechanisms with convolution operations to ensure global consistency and local feature fidelity. Moreover, GalCatDiff uses category embeddings for class-specific galaxy generation, avoiding the high computational costs of training separate models for each category. Our experimental results demonstrate that GalCatDiff significantly outperforms existing methods in terms of the consistency of sample color and size distributions, and the generated galaxies are both visually realistic and physically consistent. This framework will enhance the reliability of galaxy simulations and can potentially serve as a data augmentor to support future galaxy classification algorithm development.