# The importance of evaluating the complete automated knowledge-based planning pipeline

arXiv: https://arxiv.org/abs/1910.14257 · HTML: https://mlsanity.com/paper/1910.14257/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Oct 2019.

## Built on by (most cited first, 3 of 3)

- [Domain Knowledge Driven 3D Dose Prediction Using Moment-Based Loss Function](https://mlsanity.com/paper/2207.03414/) (2022)
- [OpenKBP-Opt: An international and reproducible evaluation of 76 knowledge-based planning pipelines](https://mlsanity.com/paper/2202.08303/) (2022)
- [Probabilistic feature extraction, dose statistic prediction and dose mimicking for automated radiation therapy treatment planning](https://mlsanity.com/paper/2102.12569/) (2021)

## Abstract (arXiv)

We determine how prediction methods combine with optimization methods in two-stage knowledge-based planning (KBP) pipelines to produce radiation therapy treatment plans. We trained two dose prediction methods, a generative adversarial network (GAN) and a random forest (RF) with the same 130 treatment plans. The models were applied to 87 out-of-sample patients to create two sets of predicted dose distributions that were used as input to two optimization models. The first optimization model, inverse planning (IP), estimates weights for dose-objectives from a predicted dose distribution and generates new plans using conventional inverse planning. The second optimization model, dose mimicking (DM), minimizes the sum of one-sided quadratic penalties between the predictions and the generated plans using several dose-objectives. Altogether, four KBP pipelines (GAN-IP, GAN-DM, RF-IP, and RF-DM) were constructed and benchmarked against the corresponding clinical plans using clinical criteria; the error of both prediction methods was also evaluated. The best performing plans were GAN-IP plans, which satisfied the same criteria as their corresponding clinical plans (78%) more often than any other KBP pipeline. However, GAN did not necessarily provide the best prediction for the second-stage optimization models. Specifically, both the RF-IP and RF-DM plans satisfied all clinical criteria 25% and 15% more often than GAN-DM plans (the worst performing planning), respectively. GAN predictions also had a higher mean absolute error (3.9 Gy) than those from RF (3.6 Gy). We find that state-of-the-art prediction methods when paired with different optimization algorithms, produce treatment plans with considerable variation in quality.