# SAM++: Enhancing Anatomic Matching using Semantic Information and Structural Inference

arXiv: https://arxiv.org/abs/2306.13988 · HTML: https://mlsanity.com/paper/2306.13988/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 20 references that are in the map.
- First posted on arXiv 24 Jun 2023.

## Builds on (most cited first, 17 of 17)

- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [SiamRPN++: Evolution of Siamese Visual Tracking with Very Deep Networks](https://mlsanity.com/paper/1812.11703/) (2018)
- [Transformer Tracking](https://mlsanity.com/paper/2103.15436/) (2021)
- [An Unsupervised Learning Model for Deformable Medical Image Registration](https://mlsanity.com/paper/1802.02604/) (2018)
- [SAM: Self-supervised Learning of Pixel-wise Anatomical Embeddings in Radiological Images](https://mlsanity.com/paper/2012.02383/) (2020)
- [Learning from Multiple Datasets with Heterogeneous and Partial Labels for Universal Lesion Detection in CT](https://mlsanity.com/paper/2009.02577/) (2020)
- [One-Shot Medical Landmark Detection](https://mlsanity.com/paper/2103.04527/) (2021)
- [An Attention-Guided Deep Regression Model for Landmark Detection in Cephalograms](https://mlsanity.com/paper/1906.07549/) (2019)
- [Deep Lesion Tracker: Monitoring Lesions in 4D Longitudinal Imaging Studies](https://mlsanity.com/paper/2012.04872/) (2020)
- [X-ray-transform Invariant Anatomical Landmark Detection for Pelvic Trauma Surgery](https://mlsanity.com/paper/1803.08608/) (2018)
- [Holistic and Comprehensive Annotation of Clinically Significant Findings on Diverse CT Images: Learning from Radiology Reports and Label Ontology](https://mlsanity.com/paper/1904.04661/) (2019)
- [Which images to label for few-shot medical landmark detection?](https://mlsanity.com/paper/2112.04386/) (2021)
- [Deep Volumetric Universal Lesion Detection using Light-Weight Pseudo 3D Convolution and Surface Point Regression](https://mlsanity.com/paper/2008.13254/) (2020)
- [Attaining human-level performance with atlas location autocontext for anatomical landmark detection in 3D CT data](https://mlsanity.com/paper/1805.08687/) (2018)
- [DATR: Domain-adaptive transformer for multi-domain landmark detection](https://mlsanity.com/paper/2203.06433/) (2022)
- [Transformer Lesion Tracker](https://mlsanity.com/paper/2206.06252/) (2022)
- [Structure-Guided Self-Supervised Matching for One-Shot Medical Landmark Detection](https://mlsanity.com/paper/2203.01687/) (2022)

## Built on by (most cited first, 5 of 5)

- [Real Time Multi Organ Classification on Computed Tomography Images](https://mlsanity.com/paper/2404.18731/) (2024)
- [Decoupling Continual Semantic Segmentation](https://mlsanity.com/paper/2508.05065/) (2025)
- [Segment Any Medical Model Extended](https://mlsanity.com/paper/2403.18114/) (2024)
- [Consistent Point Matching](https://mlsanity.com/paper/2507.23609/) (2025)
- [Distilled Roads: Generalisable Road Network Extraction Across Sensors, Resolutions, and Region](https://mlsanity.com/paper/2608.03407/) (2026)

## Abstract (arXiv)

Medical images like CT and MRI provide detailed information about the internal structure of the body, and identifying key anatomical structures from these images plays a crucial role in clinical workflows. Current methods treat it as a registration or key-point regression task, which has limitations in accurate matching and can only handle predefined landmarks. Recently, some methods have been introduced to address these limitations. One such method, called SAM, proposes using a dense self-supervised approach to learn a distinct embedding for each point on the CT image and achieving promising results. Nonetheless, SAM may still face difficulties when dealing with structures that have similar appearances but different semantic meanings or similar semantic meanings but different appearances. To overcome these limitations, we propose SAM++, a framework that simultaneously learns appearance and semantic embeddings with a novel fixed-points matching mechanism. We tested the SAM++ framework on two challenging tasks, demonstrating a significant improvement over the performance of SAM and outperforming other existing methods.