# Point detection through multi-instance deep heatmap regression for sutures in endoscopy

arXiv: https://arxiv.org/abs/2111.08468 · HTML: https://mlsanity.com/paper/2111.08468/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 31 references that are in the map.
- First posted on arXiv 16 Nov 2021.

## Builds on (most cited first, 14 of 14)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Kornia: an Open Source Differentiable Computer Vision Library for PyTorch](https://mlsanity.com/paper/1910.02190/) (2019)
- [Hand Pose Estimation via Latent 2.5D Heatmap Regression](https://mlsanity.com/paper/1804.09534/) (2018)
- [Integrating Spatial Configuration into Heatmap Regression Based CNNs for Landmark Localization](https://mlsanity.com/paper/1908.00748/) (2019)
- [Human pose estimation via Convolutional Part Heatmap Regression](https://mlsanity.com/paper/1609.01743/) (2016)
- [Deep Alignment Network: A convolutional neural network for robust face alignment](https://mlsanity.com/paper/1706.01789/) (2017)
- [Domain-adversarial neural networks to address the appearance variability of histopathology images](https://mlsanity.com/paper/1707.06183/) (2017)
- [Asymmetric Loss Functions and Deep Densely Connected Networks for Highly Imbalanced Medical Image Segmentation: Application to Multiple Sclerosis Lesion Detection](https://mlsanity.com/paper/1803.11078/) (2018)
- [Mutually improved endoscopic image synthesis and landmark detection in unpaired image-to-image translation](https://mlsanity.com/paper/2107.06941/) (2021)
- [Improving Surgical Training Phantoms by Hyperrealism: Deep Unpaired Image-to-Image Translation from Real Surgeries](https://mlsanity.com/paper/1806.03627/) (2018)
- [Flexible and Comprehensive Patient-Specific Mitral Valve Silicone Models with Chordae Tendinae Made From 3D-Printable Molds](https://mlsanity.com/paper/1904.03704/) (2019)
- [Cross-Domain Conditional Generative Adversarial Networks for Stereoscopic Hyperrealism in Surgical Training](https://mlsanity.com/paper/1906.10011/) (2019)
- [Heatmap-based 2D Landmark Detection with a Varying Number of Landmarks](https://mlsanity.com/paper/2101.02737/) (2021)
- [Globally Tuned Cascade Pose Regression via Back Propagation with Application in 2D Face Pose Estimation and Heart Segmentation in 3D CT Images](https://mlsanity.com/paper/1503.08843/) (2015)

## Built on by (most cited first, 4 of 4)

- [landmarker: a Toolkit for Anatomical Landmark Localization in 2D/3D Images](https://mlsanity.com/paper/2501.10098/) (2025)
- [Heatmap Regression for Lesion Detection using Pointwise Annotations](https://mlsanity.com/paper/2208.05939/) (2022)
- [Comparison of Depth Estimation Setups from Stereo Endoscopy and Optical Tracking for Point Measurements](https://mlsanity.com/paper/2201.10848/) (2022)
- [mvHOTA: A multi-view higher order tracking accuracy metric to measure spatial and temporal associations in multi-point detection](https://mlsanity.com/paper/2206.09372/) (2022)

## Abstract (arXiv)

Purpose: Mitral valve repair is a complex minimally invasive surgery of the heart valve. In this context, suture detection from endoscopic images is a highly relevant task that provides quantitative information to analyse suturing patterns, assess prosthetic configurations and produce augmented reality visualisations. Facial or anatomical landmark detection tasks typically contain a fixed number of landmarks, and use regression or fixed heatmap-based approaches to localize the landmarks. However in endoscopy, there are a varying number of sutures in every image, and the sutures may occur at any location in the annulus, as they are not semantically unique. Method: In this work, we formulate the suture detection task as a multi-instance deep heatmap regression problem, to identify entry and exit points of sutures. We extend our previous work, and introduce the novel use of a 2D Gaussian layer followed by a differentiable 2D spatial Soft-Argmax layer to function as a local non-maximum suppression. Results: We present extensive experiments with multiple heatmap distribution functions and two variants of the proposed model. In the intra-operative domain, Variant 1 showed a mean F1 of +0.0422 over the baseline. Similarly, in the simulator domain, Variant 1 showed a mean F1 of +0.0865 over the baseline. Conclusion: The proposed model shows an improvement over the baseline in the intra-operative and the simulator domains. The data is made publicly available within the scope of the MICCAI AdaptOR2021 Challenge https://adaptor2021.github.io/, and the code at https://github.com/Cardio-AI/suture-detection-pytorch/. DOI:10.1007/s11548-021-02523-w. The link to the open access article can be found here: https://link.springer.com/article/10.1007%2Fs11548-021-02523-w