# Landmark Tracking in Liver US images Using Cascade Convolutional Neural Networks with Long Short-Term Memory

arXiv: https://arxiv.org/abs/2209.06952 · HTML: https://mlsanity.com/paper/2209.06952/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 47 references that are in the map.
- First posted on arXiv 14 Sep 2022.

## Builds on (most cited first, 13 of 13)

- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [LSTM: A Search Space Odyssey](https://mlsanity.com/paper/1503.04069/) (2015)
- [Large-Margin Softmax Loss for Convolutional Neural Networks](https://mlsanity.com/paper/1612.02295/) (2016)
- [PRNet: Self-Supervised Learning for Partial-to-Partial Registration](https://mlsanity.com/paper/1910.12240/) (2019)
- [Large Margin Deep Networks for Classification](https://mlsanity.com/paper/1803.05598/) (2018)
- [Deep Learning in Medical Image Registration: A Review](https://mlsanity.com/paper/1912.12318/) (2019)
- [Hybrid LSTM and Encoder-Decoder Architecture for Detection of Image Forgeries](https://mlsanity.com/paper/1903.02495/) (2019)
- [Deep learning in ultrasound imaging](https://mlsanity.com/paper/1907.02994/) (2019)
- [DeformRS: Certifying Input Deformations with Randomized Smoothing](https://mlsanity.com/paper/2107.00996/) (2021)
- [Siamese Networks with Location Prior for Landmark Tracking in Liver Ultrasound Sequences](https://mlsanity.com/paper/1901.08109/) (2019)
- [Mask R-CNN with Pyramid Attention Network for Scene Text Detection](https://mlsanity.com/paper/1811.09058/) (2018)
- [Assisted Probe Positioning for Ultrasound Guided Radiotherapy Using Image Sequence Classification](https://mlsanity.com/paper/2010.02732/) (2020)

## Built on by (most cited first, 2 of 2)

- [CatNet: Controlling the False Discovery Rate in LSTM with SHAP Feature Importance and Gaussian Mirrors](https://mlsanity.com/paper/2411.16666/) (2024)
- [Model-Agnostic FDR Control via Group Gaussian Mirror and Permutation SHAP](https://mlsanity.com/paper/2608.00989/) (2026)

## Abstract (arXiv)

This study proposed a deep learning-based tracking method for ultrasound (US) image-guided radiation therapy. The proposed cascade deep learning model is composed of an attention network, a mask region-based convolutional neural network (mask R-CNN), and a long short-term memory (LSTM) network. The attention network learns a mapping from a US image to a suspected area of landmark motion in order to reduce the search region. The mask R-CNN then produces multiple region-of-interest (ROI) proposals in the reduced region and identifies the proposed landmark via three network heads: bounding box regression, proposal classification, and landmark segmentation. The LSTM network models the temporal relationship among the successive image frames for bounding box regression and proposal classification. To consolidate the final proposal, a selection method is designed according to the similarities between sequential frames. The proposed method was tested on the liver US tracking datasets used in the Medical Image Computing and Computer Assisted Interventions (MICCAI) 2015 challenges, where the landmarks were annotated by three experienced observers to obtain their mean positions. Five-fold cross-validation on the 24 given US sequences with ground truths shows that the mean tracking error for all landmarks is 0.65+/-0.56 mm, and the errors of all landmarks are within 2 mm. We further tested the proposed model on 69 landmarks from the testing dataset that has a similar image pattern to the training pattern, resulting in a mean tracking error of 0.94+/-0.83 mm. Our experimental results have demonstrated the feasibility and accuracy of our proposed method in tracking liver anatomic landmarks using US images, providing a potential solution for real-time liver tracking for active motion management during radiation therapy.