# Multi-Target Landmark Detection with Incomplete Images via Reinforcement Learning and Shape Prior

arXiv: https://arxiv.org/abs/2301.05392 · HTML: https://mlsanity.com/paper/2301.05392/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 80 references that are in the map.
- First posted on arXiv 13 Jan 2023.

## Builds on (most cited first, 18 of 18)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks](https://mlsanity.com/paper/1604.02878/) (2016)
- [Multivariate mixture model for myocardium segmentation combining multi-source images](https://mlsanity.com/paper/1612.08820/) (2016)
- [Evaluation of Algorithms for Multi-Modality Whole Heart Segmentation: An Open-Access Grand Challenge](https://mlsanity.com/paper/1902.07880/) (2019)
- [Facial Landmark Detection: a Literature Survey](https://mlsanity.com/paper/1805.05563/) (2018)
- [Cardiac Segmentation on Late Gadolinium Enhancement MRI: A Benchmark Study from Multi-Sequence Cardiac MR Segmentation Challenge](https://mlsanity.com/paper/2006.12434/) (2020)
- [Deep reinforcement learning in medical imaging: A literature review](https://mlsanity.com/paper/2103.05115/) (2021)
- [DeepSSM: A Deep Learning Framework for Statistical Shape Modeling from Raw Images](https://mlsanity.com/paper/1810.00111/) (2018)
- [Cardiac Segmentation from LGE MRI Using Deep Neural Network Incorporating Shape and Spatial Priors](https://mlsanity.com/paper/1906.07347/) (2019)
- [Deep Implicit Statistical Shape Models for 3D Medical Image Delineation](https://mlsanity.com/paper/2104.02847/) (2021)
- [An Artificial Agent for Robust Image Registration](https://mlsanity.com/paper/1611.10336/) (2016)
- [Multiple Landmark Detection using Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1907.00318/) (2019)
- [Automatic View Planning with Multi-scale Deep Reinforcement Learning Agents](https://mlsanity.com/paper/1806.03228/) (2018)
- [Atrial Scar Quantification via Multi-scale CNN in the Graph-cuts Framework](https://mlsanity.com/paper/1902.07877/) (2019)
- [Enhanced detection of fetal pose in 3D MRI by Deep Reinforcement Learning with physical structure priors on anatomy](https://mlsanity.com/paper/2007.08146/) (2020)
- [Smooth Extrapolation of Unknown Anatomy via Statistical Shape Models](https://mlsanity.com/paper/1909.10153/) (2019)
- [Communicative Reinforcement Learning Agents for Landmark Detection in Brain Images](https://mlsanity.com/paper/2008.08055/) (2020)
- [Multi-dimensional Gated Recurrent Units for Automated Anatomical Landmark Localization](https://mlsanity.com/paper/1708.02766/) (2017)

## Built on by (most cited first, 2 of 2)

- [Personalized Topology-Informed Localization of Standard 12-Lead ECG Electrode Placement from Incomplete Cardiac MRIs for Efficient Cardiac Digital Twins](https://mlsanity.com/paper/2408.13945/) (2024)
- [Comprehensive Review of Reinforcement Learning for Medical Ultrasound Imaging](https://mlsanity.com/paper/2503.16543/) (2025)

## Abstract (arXiv)

Medical images are generally acquired with limited field-of-view (FOV), which could lead to incomplete regions of interest (ROI), and thus impose a great challenge on medical image analysis. This is particularly evident for the learning-based multi-target landmark detection, where algorithms could be misleading to learn primarily the variation of background due to the varying FOV, failing the detection of targets. Based on learning a navigation policy, instead of predicting targets directly, reinforcement learning (RL)-based methods have the potential totackle this challenge in an efficient manner. Inspired by this, in this work we propose a multi-agent RL framework for simultaneous multi-target landmark detection. This framework is aimed to learn from incomplete or (and) complete images to form an implicit knowledge of global structure, which is consolidated during the training stage for the detection of targets from either complete or incomplete test images. To further explicitly exploit the global structural information from incomplete images, we propose to embed a shape model into the RL process. With this prior knowledge, the proposed RL model can not only localize dozens of targetssimultaneously, but also work effectively and robustly in the presence of incomplete images. We validated the applicability and efficacy of the proposed method on various multi-target detection tasks with incomplete images from practical clinics, using body dual-energy X-ray absorptiometry (DXA), cardiac MRI and head CT datasets. Results showed that our method could predict whole set of landmarks with incomplete training images up to 80% missing proportion (average distance error 2.29 cm on body DXA), and could detect unseen landmarks in regions with missing image information outside FOV of target images (average distance error 6.84 mm on 3D half-head CT).