# From Pixel to Cancer: Cellular Automata in Computed Tomography

arXiv: https://arxiv.org/abs/2403.06459 · HTML: https://mlsanity.com/paper/2403.06459/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 64 references that are in the map.
- First posted on arXiv 11 Mar 2024.

## Builds on (most cited first, 27 of 27)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning](https://mlsanity.com/paper/1612.06890/) (2016)
- [UNETR: Transformers for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.10504/) (2021)
- [Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images](https://mlsanity.com/paper/2201.01266/) (2022)
- [UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation](https://mlsanity.com/paper/1912.05074/) (2019)
- [The Liver Tumor Segmentation Benchmark (LiTS)](https://mlsanity.com/paper/1901.04056/) (2019)
- [A large annotated medical image dataset for the development and evaluation of segmentation algorithms](https://mlsanity.com/paper/1902.09063/) (2019)
- [CHAOS Challenge -- Combined (CT-MR) Healthy Abdominal Organ Segmentation](https://mlsanity.com/paper/2001.06535/) (2020)
- [AbdomenCT-1K: Is Abdominal Organ Segmentation A Solved Problem?](https://mlsanity.com/paper/2010.14808/) (2020)
- [CLIP-Driven Universal Model for Organ Segmentation and Tumor Detection](https://mlsanity.com/paper/2301.00785/) (2023)
- [The KiTS19 Challenge Data: 300 Kidney Tumor Cases with Clinical Context, CT Semantic Segmentations, and Surgical Outcomes](https://mlsanity.com/paper/1904.00445/) (2019)
- [Transferable Visual Words: Exploiting the Semantics of Anatomical Patterns for Self-supervised Learning](https://mlsanity.com/paper/2102.10680/) (2021)
- [Synthetic Data -- what, why and how?](https://mlsanity.com/paper/2205.03257/) (2022)
- [Towards Generalizable Tumor Synthesis](https://mlsanity.com/paper/2402.19470/) (2024)
- [Label-Free Liver Tumor Segmentation](https://mlsanity.com/paper/2303.14869/) (2023)
- [Label-Free Segmentation of COVID-19 Lesions in Lung CT](https://mlsanity.com/paper/2009.06456/) (2020)
- [Synthesizing Diverse Lung Nodules Wherever Massively: 3D Multi-Conditional GAN-based CT Image Augmentation for Object Detection](https://mlsanity.com/paper/1906.04962/) (2019)
- [Privacy Distillation: Reducing Re-identification Risk of Multimodal Diffusion Models](https://mlsanity.com/paper/2306.01322/) (2023)
- [Early Detection and Localization of Pancreatic Cancer by Label-Free Tumor Synthesis](https://mlsanity.com/paper/2308.03008/) (2023)
- [Boosting Dermatoscopic Lesion Segmentation via Diffusion Models with Visual and Textual Prompts](https://mlsanity.com/paper/2310.02906/) (2023)
- [Free-form tumor synthesis in computed tomography images via richer generative adversarial network](https://mlsanity.com/paper/2104.09701/) (2021)
- [Synthetic Tumors Make AI Segment Tumors Better](https://mlsanity.com/paper/2210.14845/) (2022)
- [PoseExaminer: Automated Testing of Out-of-Distribution Robustness in Human Pose and Shape Estimation](https://mlsanity.com/paper/2303.07337/) (2023)
- [Synthetic Data as Validation](https://mlsanity.com/paper/2310.16052/) (2023)
- [Less is More: Unsupervised Mask-guided Annotated CT Image Synthesis with Minimum Manual Segmentations](https://mlsanity.com/paper/2303.12747/) (2023)
- [How Good Are Synthetic Medical Images? An Empirical Study with Lung Ultrasound](https://mlsanity.com/paper/2310.03608/) (2023)

## Built on by (most cited first, 20 of 20)

- [MAISI: Medical AI for Synthetic Imaging](https://mlsanity.com/paper/2409.11169/) (2024)
- [AbdomenAtlas: A Large-Scale, Detailed-Annotated, & Multi-Center Dataset for Efficient Transfer Learning and Open Algorithmic Benchmarking](https://mlsanity.com/paper/2407.16697/) (2024)
- [Medical World Model: Generative Simulation of Tumor Evolution for Treatment Planning](https://mlsanity.com/paper/2506.02327/) (2025)
- [Universal and Extensible Language-Vision Models for Organ Segmentation and Tumor Detection from Abdominal Computed Tomography](https://mlsanity.com/paper/2405.18356/) (2024)
- [Touchstone Benchmark: Are We on the Right Way for Evaluating AI Algorithms for Medical Segmentation?](https://mlsanity.com/paper/2411.03670/) (2024)
- [LeFusion: Controllable Pathology Synthesis via Lesion-Focused Diffusion Models](https://mlsanity.com/paper/2403.14066/) (2024)
- [FreeTumor: Advance Tumor Segmentation via Large-Scale Tumor Synthesis](https://mlsanity.com/paper/2406.01264/) (2024)
- [Text-Driven Tumor Synthesis](https://mlsanity.com/paper/2412.18589/) (2024)
- [Scaling Tumor Segmentation: Best Lessons from Real and Synthetic Data](https://mlsanity.com/paper/2510.14831/) (2025)
- [Analyzing Tumors by Synthesis](https://mlsanity.com/paper/2409.06035/) (2024)
- [Embracing Massive Medical Data](https://mlsanity.com/paper/2407.04687/) (2024)
- [Early and Prediagnostic Detection of Pancreatic Cancer from Computed Tomography](https://mlsanity.com/paper/2601.22134/) (2026)
- [Optimizing Synthetic Data for Enhanced Pancreatic Tumor Segmentation](https://mlsanity.com/paper/2407.19284/) (2024)
- [See More, Change Less: Anatomy-Aware Diffusion for Contrast Enhancement](https://mlsanity.com/paper/2512.07251/) (2025)
- [Are Video Models Emerging as Zero-Shot Learners and Reasoners in Medical Imaging?](https://mlsanity.com/paper/2510.10254/) (2025)
- [3D-LLDM: Label-Guided 3D Latent Diffusion Model for Improving High-Resolution Synthetic MR Imaging in Hepatic Structure Segmentation](https://mlsanity.com/paper/2603.23845/) (2026)
- [Interactive Segmentation and Report Generation for CT Images](https://mlsanity.com/paper/2503.03294/) (2025)
- [Patient-Specific Autoregressive Models for Organ Motion Prediction in Radiotherapy](https://mlsanity.com/paper/2505.11832/) (2025)
- [ShapeKit](https://mlsanity.com/paper/2506.24003/) (2025)
- [Beyond Masks: The Case for Medical Image Parsing](https://mlsanity.com/paper/2605.11438/) (2026)

## Abstract (arXiv)

AI for cancer detection encounters the bottleneck of data scarcity, annotation difficulty, and low prevalence of early tumors. Tumor synthesis seeks to create artificial tumors in medical images, which can greatly diversify the data and annotations for AI training. However, current tumor synthesis approaches are not applicable across different organs due to their need for specific expertise and design. This paper establishes a set of generic rules to simulate tumor development. Each cell (pixel) is initially assigned a state between zero and ten to represent the tumor population, and a tumor can be developed based on three rules to describe the process of growth, invasion, and death. We apply these three generic rules to simulate tumor development--from pixel to cancer--using cellular automata. We then integrate the tumor state into the original computed tomography (CT) images to generate synthetic tumors across different organs. This tumor synthesis approach allows for sampling tumors at multiple stages and analyzing tumor-organ interaction. Clinically, a reader study involving three expert radiologists reveals that the synthetic tumors and their developing trajectories are convincingly realistic. Technically, we analyze and simulate tumor development at various stages using 9,262 raw, unlabeled CT images sourced from 68 hospitals worldwide. The performance in segmenting tumors in the liver, pancreas, and kidneys exceeds prevailing literature benchmarks, underlining the immense potential of tumor synthesis, especially for earlier cancer detection. The code and models are available at https://github.com/MrGiovanni/Pixel2Cancer