# DenseSeg: Joint Learning for Semantic Segmentation and Landmark Detection Using Dense Image-to-Shape Representation

arXiv: https://arxiv.org/abs/2405.19746 · HTML: https://mlsanity.com/paper/2405.19746/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 28 references that are in the map.
- First posted on arXiv 30 May 2024.

## Builds on (most cited first, 19 of 19)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Point Transformer](https://mlsanity.com/paper/2011.00931/) (2020)
- [DensePose: Dense Human Pose Estimation In The Wild](https://mlsanity.com/paper/1802.00434/) (2018)
- [Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation](https://mlsanity.com/paper/1901.02970/) (2019)
- [PolarMask: Single Shot Instance Segmentation with Polar Representation](https://mlsanity.com/paper/1909.13226/) (2019)
- [Neural Body Fitting: Unifying Deep Learning and Model-Based Human Pose and Shape Estimation](https://mlsanity.com/paper/1808.05942/) (2018)
- [Cell Detection with Star-convex Polygons](https://mlsanity.com/paper/1806.03535/) (2018)
- [Gated-SCNN: Gated Shape CNNs for Semantic Segmentation](https://mlsanity.com/paper/1907.05740/) (2019)
- [HyperFace: A Deep Multi-task Learning Framework for Face Detection, Landmark Localization, Pose Estimation, and Gender Recognition](https://mlsanity.com/paper/1603.01249/) (2016)
- [PolyTransform: Deep Polygon Transformer for Instance Segmentation](https://mlsanity.com/paper/1912.02801/) (2019)
- [Human pose estimation via Convolutional Part Heatmap Regression](https://mlsanity.com/paper/1609.01743/) (2016)
- [DenseReg: Fully Convolutional Dense Shape Regression In-the-Wild](https://mlsanity.com/paper/1803.02188/) (2018)
- [Improving anatomical plausibility in medical image segmentation via hybrid graph neural networks: applications to chest x-ray analysis](https://mlsanity.com/paper/2203.10977/) (2022)
- [Poly-YOLO: higher speed, more precise detection and instance segmentation for YOLOv3](https://mlsanity.com/paper/2005.13243/) (2020)
- [A CNN Cascade for Landmark Guided Semantic Part Segmentation](https://mlsanity.com/paper/1609.09642/) (2016)
- [Unsupervised learning of object frames by dense equivariant image labelling](https://mlsanity.com/paper/1706.02932/) (2017)
- [Combining Image- and Geometric-based Deep Learning for Shape Regression: A Comparison to Pixel-level Methods for Segmentation in Chest X-Ray](https://mlsanity.com/paper/2401.07542/) (2024)

## Built on by (most cited first, 1 of 1)

- [Shaken or Stirred? An Analysis of MetaFormer's Token Mixing for Medical Imaging](https://mlsanity.com/paper/2510.05971/) (2025)

## Abstract (arXiv)

Purpose: Semantic segmentation and landmark detection are fundamental tasks of medical image processing, facilitating further analysis of anatomical objects. Although deep learning-based pixel-wise classification has set a new-state-of-the-art for segmentation, it falls short in landmark detection, a strength of shape-based approaches. Methods: In this work, we propose a dense image-to-shape representation that enables the joint learning of landmarks and semantic segmentation by employing a fully convolutional architecture. Our method intuitively allows the extraction of arbitrary landmarks due to its representation of anatomical correspondences. We benchmark our method against the state-of-the-art for semantic segmentation (nnUNet), a shape-based approach employing geometric deep learning and a convolutional neural network-based method for landmark detection. Results: We evaluate our method on two medical dataset: one common benchmark featuring the lungs, heart, and clavicle from thorax X-rays, and another with 17 different bones in the paediatric wrist. While our method is on pair with the landmark detection baseline in the thorax setting (error in mm of 2.6±0.9 vs 2.7±0.9), it substantially surpassed it in the more complex wrist setting (1.1±0.6 vs 1.9±0.5). Conclusion: We demonstrate that dense geometric shape representation is beneficial for challenging landmark detection tasks and outperforms previous state-of-the-art using heatmap regression. While it does not require explicit training on the landmarks themselves, allowing for the addition of new landmarks without necessitating retraining.