# Heatmap-based 2D Landmark Detection with a Varying Number of Landmarks

arXiv: https://arxiv.org/abs/2101.02737 · HTML: https://mlsanity.com/paper/2101.02737/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 11 references that are in the map.
- First posted on arXiv 7 Jan 2021.

## Builds on (most cited first, 6 of 6)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Pixel-in-Pixel Net: Towards Efficient Facial Landmark Detection in the Wild](https://mlsanity.com/paper/2003.03771/) (2020)
- [Deep Learning-Based Regression and Classification for Automatic Landmark Localization in Medical Images](https://mlsanity.com/paper/2007.05295/) (2020)
- [Improving Surgical Training Phantoms by Hyperrealism: Deep Unpaired Image-to-Image Translation from Real Surgeries](https://mlsanity.com/paper/1806.03627/) (2018)
- [Automated Left Ventricle Dimension Measurement in 2D Cardiac Ultrasound via an Anatomically Meaningful CNN Approach](https://mlsanity.com/paper/1911.02448/) (2019)
- [Flexible and Comprehensive Patient-Specific Mitral Valve Silicone Models with Chordae Tendinae Made From 3D-Printable Molds](https://mlsanity.com/paper/1904.03704/) (2019)

## Built on by (most cited first, 5 of 5)

- [Mutually improved endoscopic image synthesis and landmark detection in unpaired image-to-image translation](https://mlsanity.com/paper/2107.06941/) (2021)
- [Point detection through multi-instance deep heatmap regression for sutures in endoscopy](https://mlsanity.com/paper/2111.08468/) (2021)
- [Heatmap Regression for Lesion Detection using Pointwise Annotations](https://mlsanity.com/paper/2208.05939/) (2022)
- [Improved Heatmap-based Landmark Detection](https://mlsanity.com/paper/2110.05676/) (2021)
- [mvHOTA: A multi-view higher order tracking accuracy metric to measure spatial and temporal associations in multi-point detection](https://mlsanity.com/paper/2206.09372/) (2022)

## Abstract (arXiv)

Mitral valve repair is a surgery to restore the function of the mitral valve. To achieve this, a prosthetic ring is sewed onto the mitral annulus. Analyzing the sutures, which are punctured through the annulus for ring implantation, can be useful in surgical skill assessment, for quantitative surgery and for positioning a virtual prosthetic ring model in the scene via augmented reality. This work presents a neural network approach which detects the sutures in endoscopic images of mitral valve repair and therefore solves a landmark detection problem with varying amount of landmarks, as opposed to most other existing deep learning-based landmark detection approaches. The neural network is trained separately on two data collections from different domains with the same architecture and hyperparameter settings. The datasets consist of more than 1,300 stereo frame pairs each, with a total over 60,000 annotated landmarks. The proposed heatmap-based neural network achieves a mean positive predictive value (PPV) of 66.68±4.67% and a mean true positive rate (TPR) of 24.45±5.06% on the intraoperative test dataset and a mean PPV of 81.50±5.77% and a mean TPR of 61.60±6.11% on a dataset recorded during surgical simulation. The best detection results are achieved when the camera is positioned above the mitral valve with good illumination. A detection from a sideward view is also possible if the mitral valve is well perceptible.