# Multi-Agent Reinforcement Learning Meets Leaf Sequencing in Radiotherapy

arXiv: https://arxiv.org/abs/2406.01853 · HTML: https://mlsanity.com/paper/2406.01853/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 66 references that are in the map.
- First posted on arXiv 3 Jun 2024.

## Builds on (most cited first, 15 of 15)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games](https://mlsanity.com/paper/2103.01955/) (2021)
- [Deep Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2002.00444/) (2020)
- [Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms](https://mlsanity.com/paper/1911.10635/) (2019)
- [Neural Policy Gradient Methods: Global Optimality and Rates of Convergence](https://mlsanity.com/paper/1909.01150/) (2019)
- [Phasic Policy Gradient](https://mlsanity.com/paper/2009.04416/) (2020)
- [Time Limits in Reinforcement Learning](https://mlsanity.com/paper/1712.00378/) (2017)
- [OpenKBP: The open-access knowledge-based planning grand challenge](https://mlsanity.com/paper/2011.14076/) (2020)
- [DiffDP: Radiotherapy Dose Prediction via a Diffusion Model](https://mlsanity.com/paper/2307.09794/) (2023)
- [Three-Dimensional Dose Prediction for Lung IMRT Patients with Deep Neural Networks: Robust Learning from Heterogeneous Beam Configurations](https://mlsanity.com/paper/1812.06934/) (2018)
- [Robust Direct Aperture Optimization for Radiation Therapy Treatment Planning](https://mlsanity.com/paper/2111.04847/) (2021)

## Built on by (most cited first, 6 of 6)

- [Automating RT Planning at Scale: High Quality Data For AI Training](https://mlsanity.com/paper/2501.11803/) (2025)
- [Demo: Generative AI helps Radiotherapy Planning with User Preference](https://mlsanity.com/paper/2512.08996/) (2025)
- [AI End-to-End Radiation Treatment Planning Under One Second](https://mlsanity.com/paper/2603.06338/) (2026)
- [Any2Any 3D Diffusion Models with Knowledge Transfer: A Radiotherapy Planning Study](https://mlsanity.com/paper/2605.09622/) (2026)
- [Towards Human-Centric Intelligent Treatment Planning for Radiation Therapy](https://mlsanity.com/paper/2510.13062/) (2025)
- [Learning to Optimize Radiotherapy Plans via Fluence Maps Diffusion Model Generation and LSTM-based Optimization](https://mlsanity.com/paper/2605.13713/) (2026)

## Abstract (arXiv)

In contemporary radiotherapy planning (RTP), a key module leaf sequencing is predominantly addressed by optimization-based approaches. In this paper, we propose a novel deep reinforcement learning (DRL) model termed as Reinforced Leaf Sequencer (RLS) in a multi-agent framework for leaf sequencing. The RLS model offers improvements to time-consuming iterative optimization steps via large-scale training and can control movement patterns through the design of reward mechanisms. We have conducted experiments on four datasets with four metrics and compared our model with a leading optimization sequencer. Our findings reveal that the proposed RLS model can achieve reduced fluence reconstruction errors, and potential faster convergence when integrated in an optimization planner. Additionally, RLS has shown promising results in a full artificial intelligence RTP pipeline. We hope this pioneer multi-agent RL leaf sequencer can foster future research on machine learning for RTP.