# Latent Spaces Enable Transformer-Based Dose Prediction in Complex Radiotherapy Plans

arXiv: https://arxiv.org/abs/2407.08650 · HTML: https://mlsanity.com/paper/2407.08650/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 30 references that are in the map.
- First posted on arXiv 11 Jul 2024.

## Builds on (most cited first, 9 of 9)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [VideoGPT: Video Generation using VQ-VAE and Transformers](https://mlsanity.com/paper/2104.10157/) (2021)
- [Jukebox: A Generative Model for Music](https://mlsanity.com/paper/2005.00341/) (2020)
- [OpenKBP: The open-access knowledge-based planning grand challenge](https://mlsanity.com/paper/2011.14076/) (2020)
- [Three-Dimensional Dose Prediction for Lung IMRT Patients with Deep Neural Networks: Robust Learning from Heterogeneous Beam Configurations](https://mlsanity.com/paper/1812.06934/) (2018)
- [OpenKBP-Opt: An international and reproducible evaluation of 76 knowledge-based planning pipelines](https://mlsanity.com/paper/2202.08303/) (2022)

## Built on by (most cited first, 1 of 1)

- [FluenceFormer: Transformer-Driven Multi-Beam Fluence Map Regression for Radiotherapy Planning](https://mlsanity.com/paper/2512.22425/) (2025)

## Abstract (arXiv)

Evidence is accumulating in favour of using stereotactic ablative body radiotherapy (SABR) to treat multiple cancer lesions in the lung. Multi-lesion lung SABR plans are complex and require significant resources to create. In this work, we propose a novel two-stage latent transformer framework (LDFormer) for dose prediction of lung SABR plans with varying numbers of lesions. In the first stage, patient anatomical information and the dose distribution are encoded into a latent space. In the second stage, a transformer learns to predict the dose latent from the anatomical latents. Causal attention is modified to adapt to different numbers of lesions. LDFormer outperforms a state-of-the-art generative adversarial network on dose conformality in and around lesions, and the performance gap widens when considering overlapping lesions. LDFormer generates predictions of 3-D dose distributions in under 30s on consumer hardware, and has the potential to assist physicians with clinical decision making, reduce resource costs, and accelerate treatment planning.