# TongueSAM: An Universal Tongue Segmentation Model Based on SAM with Zero-Shot

arXiv: https://arxiv.org/abs/2308.06444 · HTML: https://mlsanity.com/paper/2308.06444/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 24 references that are in the map.
- First posted on arXiv 12 Aug 2023.

## Builds on (most cited first, 9 of 9)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)
- [Segment Anything Model for Medical Image Analysis: an Experimental Study](https://mlsanity.com/paper/2304.10517/) (2023)
- [Generalist Vision Foundation Models for Medical Imaging: A Case Study of Segment Anything Model on Zero-Shot Medical Segmentation](https://mlsanity.com/paper/2304.12637/) (2023)
- [SkinSAM: Empowering Skin Cancer Segmentation with Segment Anything Model](https://mlsanity.com/paper/2304.13973/) (2023)
- [Zero-shot performance of the Segment Anything Model (SAM) in 2D medical imaging: A comprehensive evaluation and practical guidelines](https://mlsanity.com/paper/2305.00109/) (2023)

## Built on by (most cited first, 7 of 7)

- [MedSAM-U: Uncertainty-Guided Auto Multi-Prompt Adaptation for Reliable MedSAM](https://mlsanity.com/paper/2409.00924/) (2024)
- [SAMCT: Segment Any CT Allowing Labor-Free Task-Indicator Prompts](https://mlsanity.com/paper/2403.13258/) (2024)
- [Memory-SAM: Human-Prompt-Free Tongue Segmentation via Retrieval-to-Prompt](https://mlsanity.com/paper/2510.15849/) (2025)
- [A Non-Invasive Interpretable NAFLD Diagnostic Method Combining TCM Tongue Features](https://mlsanity.com/paper/2309.02959/) (2023)
- [TOM: An Open-Source Tongue Segmentation Method with Multi-Teacher Distillation and Task-Specific Data Augmentation](https://mlsanity.com/paper/2508.14932/) (2025)
- [MMIR-TCM: Memory-Integrated Multimodal Inference and Retrieval for TCM Clinical Decision Support](https://mlsanity.com/paper/2607.01814/) (2026)
- [TongueReenact: Geometry-Anchored Tongue Synthesis for Face Reenactment](https://mlsanity.com/paper/2607.28039/) (2026)

## Abstract (arXiv)

Tongue segmentation serves as the primary step in automated TCM tongue diagnosis, which plays a significant role in the diagnostic results. Currently, numerous deep learning based methods have achieved promising results. However, when confronted with tongue images that differ from the training set or possess challenging backgrounds, these methods demonstrate limited performance. To address this issue, this paper proposes a universal tongue segmentation model named TongueSAM based on SAM (Segment Anything Model). SAM is a large-scale pretrained interactive segmentation model known for its powerful zero-shot generalization capability. Applying SAM to tongue segmentation leverages its learned prior knowledge from natural images, enabling the achievement of zero-shot segmentation for various types of tongue images. In this study, a Prompt Generator based on object detection is integrated into SAM to enable an end-to-end automated tongue segmentation method. Experiments demonstrate that TongueSAM achieves exceptional performance across various of tongue segmentation datasets, particularly under zero-shot. Even when dealing with challenging background tongue images, TongueSAM achieves a mIoU of 95.23% under zero-shot conditions, surpassing other segmentation methods. As far as we know, this is the first application of large-scale pretrained model for tongue segmentation. The project mentioned in this paper is currently publicly available.