# Frequency-Time Diffusion with Neural Cellular Automata

arXiv: https://arxiv.org/abs/2401.06291 · HTML: https://mlsanity.com/paper/2401.06291/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 26 of its 37 references that are in the map.
- First posted on arXiv 11 Jan 2024.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Image Super-Resolution via Iterative Refinement](https://mlsanity.com/paper/2104.07636/) (2021)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [Cascaded Diffusion Models for High Fidelity Image Generation](https://mlsanity.com/paper/2106.15282/) (2021)
- [Diffusion Models in Vision: A Survey](https://mlsanity.com/paper/2209.04747/) (2022)
- [On Distillation of Guided Diffusion Models](https://mlsanity.com/paper/2210.03142/) (2022)
- [Simple diffusion: End-to-end diffusion for high resolution images](https://mlsanity.com/paper/2301.11093/) (2023)
- [MDTv2: Masked Diffusion Transformer is a Strong Image Synthesizer](https://mlsanity.com/paper/2303.14389/) (2023)
- [Cellular automata as convolutional neural networks](https://mlsanity.com/paper/1809.02942/) (2018)
- [A Theoretical Justification for Image Inpainting using Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2302.01217/) (2023)
- [Variational Neural Cellular Automata](https://mlsanity.com/paper/2201.12360/) (2022)
- [Med-NCA: Robust and Lightweight Segmentation with Neural Cellular Automata](https://mlsanity.com/paper/2302.03473/) (2023)
- [DyNCA: Real-time Dynamic Texture Synthesis Using Neural Cellular Automata](https://mlsanity.com/paper/2211.11417/) (2022)
- [Generative Adversarial Neural Cellular Automata](https://mlsanity.com/paper/2108.04328/) (2021)
- [Texture Generation with Neural Cellular Automata](https://mlsanity.com/paper/2105.07299/) (2021)

## Built on by (most cited first, 9 of 9)

- [Deep Generative Models for 3D Medical Image Synthesis](https://mlsanity.com/paper/2410.17664/) (2024)
- [Neural Cellular Automata: From Cells to Pixels](https://mlsanity.com/paper/2506.22899/) (2025)
- [AdaNCA: Neural Cellular Automata As Adaptors For More Robust Vision Transformer](https://mlsanity.com/paper/2406.08298/) (2024)
- [An Organism Starts with a Single Pix-Cell: A Neural Cellular Diffusion for High-Resolution Image Synthesis](https://mlsanity.com/paper/2407.03018/) (2024)
- [A New Kind of Network? Review and Reference Implementation of Neural Cellular Automata](https://mlsanity.com/paper/2604.24990/) (2026)
- [MedSegDiffNCA: Diffusion Models With Neural Cellular Automata for Skin Lesion Segmentation](https://mlsanity.com/paper/2501.02447/) (2025)
- [Applications of Neural Cellular Automata: State of the Art, Challenges and Opportunities](https://mlsanity.com/paper/2609.24595/) (2026)
- [Why Do Conventional World Models Fail to Learn Cellular Automata?](https://mlsanity.com/paper/2609.39604/) (2026)
- [2D Spatial Reasoning with Adaptive Neural Cellular Automata](https://mlsanity.com/paper/2610.08518/) (2026)

## Abstract (arXiv)

Despite considerable success, large Denoising Diffusion Models (DDMs) with UNet backbone pose practical challenges, particularly on limited hardware and in processing gigapixel images. To address these limitations, we introduce two Neural Cellular Automata (NCA)-based DDMs: Diff-NCA and FourierDiff-NCA. Capitalizing on the local communication capabilities of NCA, Diff-NCA significantly reduces the parameter counts of NCA-based DDMs. Integrating Fourier-based diffusion enables global communication early in the diffusion process. This feature is particularly valuable in synthesizing complex images with important global features, such as the CelebA dataset. We demonstrate that even a 331k parameter Diff-NCA can generate 512x512 pathology slices, while FourierDiff-NCA (1.1m parameters) reaches a three times lower FID score of 43.86, compared to the four times bigger UNet (3.94m parameters) with a score of 128.2. Additionally, FourierDiff-NCA can perform diverse tasks such as super-resolution, out-of-distribution image synthesis, and inpainting without explicit training.