# Surgical Anatomy Recognition with Context Learning using Foundation Representations

arXiv: https://arxiv.org/abs/2606.22124 · HTML: https://mlsanity.com/paper/2606.22124/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 26 references that are in the map.
- First posted on arXiv 20 Jun 2026.

## Builds on (most cited first, 17 of 17)

- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videos](https://mlsanity.com/paper/1602.03012/) (2016)
- [Putting the Object Back into Video Object Segmentation](https://mlsanity.com/paper/2310.12982/) (2023)
- [2017 Robotic Instrument Segmentation Challenge](https://mlsanity.com/paper/1902.06426/) (2019)
- [CholecSeg8k: A Semantic Segmentation Dataset for Laparoscopic Cholecystectomy Based on Cholec80](https://mlsanity.com/paper/2012.12453/) (2020)
- [Surgical Data Science -- from Concepts toward Clinical Translation](https://mlsanity.com/paper/2011.02284/) (2020)
- [Foundation Model for Endoscopy Video Analysis via Large-scale Self-supervised Pre-train](https://mlsanity.com/paper/2306.16741/) (2023)
- [SAM2-UNet: Segment Anything 2 Makes Strong Encoder for Natural and Medical Image Segmentation](https://mlsanity.com/paper/2408.08870/) (2024)
- [Your ViT is Secretly an Image Segmentation Model](https://mlsanity.com/paper/2503.19108/) (2025)
- [Heidelberg Colorectal Data Set for Surgical Data Science in the Sensor Operating Room](https://mlsanity.com/paper/2005.03501/) (2020)
- [General surgery vision transformer: A video pre-trained foundation model for general surgery](https://mlsanity.com/paper/2403.05949/) (2024)
- [Scaling up self-supervised learning for improved surgical foundation models](https://mlsanity.com/paper/2501.09436/) (2025)
- [The Endoscapes Dataset for Surgical Scene Segmentation, Object Detection, and Critical View of Safety Assessment: Official Splits and Benchmark](https://mlsanity.com/paper/2312.12429/) (2023)
- [LEMON: A Large Endoscopic MONocular Dataset and Foundation Model for Perception in Surgical Settings](https://mlsanity.com/paper/2503.19740/) (2025)
- [VidEoMT: Your ViT is Secretly Also a Video Segmentation Model](https://mlsanity.com/paper/2602.17807/) (2026)
- [SAM3-UNet: Simplified Adaptation of Segment Anything Model 3](https://mlsanity.com/paper/2512.01789/) (2025)
- [Benchmarking Pretrained Attention-based Models for Real-Time Recognition in Robot-Assisted Esophagectomy](https://mlsanity.com/paper/2412.03401/) (2024)

## Built on by (most cited first, 1 of 1)

- [A Surgical Foundation Model Reveals Task-Dependent Label Efficiency](https://mlsanity.com/paper/2609.31821/) (2026)

## Abstract (arXiv)

Accurate recognition of anatomical structures is essential for safe and effective minimally invasive surgery (MIS), yet it remains underexplored in surgical computer vision due to limited annotated data and methods tailored primarily to natural scenes. In this work, we present a combined dataset and model framework to advance anatomy-aware perception in MIS. First, we introduce ATLAS-120k, a large-scale clip-level semantic segmentation dataset comprising over 120,000 annotated frames from 100 surgical videos spanning 14 procedures and multiple modalities, including laparoscopic and robot-assisted surgery. The dataset captures substantial procedural variability and was created using a scalable annotation pipeline that integrates expert manual labeling, automated propagation, iterative refinement, and surgeon verification to ensure high-quality annotations. Second, we propose ATLAS (Anatomy Recognition with Context Learning using Foundation Representations), a video semantic segmentation model specifically designed for surgical anatomy recognition. Unlike conventional approaches that emphasize object tracking, ATLAS leverages foundation-model embeddings together with lightweight temporal reasoning to incorporate contextual cues such as procedure type, surgical phase, and short-term visual memory. This design enables temporally consistent and accurate predictions while maintaining real-time feasibility. Together, the dataset and model establish a practical foundation for robust surgical scene understanding and support the development of clinically applicable guidance systems for minimally invasive surgery. The models, dataset annotations and annotation platform are publicly available at: https://github.com/TimJaspers0801/ATLAS.