# SurgT challenge: Benchmark of Soft-Tissue Trackers for Robotic Surgery

arXiv: https://arxiv.org/abs/2302.03022 · HTML: https://mlsanity.com/paper/2302.03022/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 44 references that are in the map.
- First posted on arXiv 6 Feb 2023.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Context Encoders: Feature Learning by Inpainting](https://mlsanity.com/paper/1604.07379/) (2016)
- [Hindsight Experience Replay](https://mlsanity.com/paper/1707.01495/) (2017)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [Transformer Tracking](https://mlsanity.com/paper/2103.15436/) (2021)
- [EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videos](https://mlsanity.com/paper/1602.03012/) (2016)
- [2017 Robotic Instrument Segmentation Challenge](https://mlsanity.com/paper/1902.06426/) (2019)
- [Learning by Analogy: Reliable Supervision from Transformations for Unsupervised Optical Flow Estimation](https://mlsanity.com/paper/2003.13045/) (2020)
- [A Novel Performance Evaluation Methodology for Single-Target Trackers](https://mlsanity.com/paper/1503.01313/) (2015)
- [Stereo Correspondence and Reconstruction of Endoscopic Data Challenge](https://mlsanity.com/paper/2101.01133/) (2021)
- [Unsupervised Deep Tracking](https://mlsanity.com/paper/1904.01828/) (2019)
- [Discriminative Correlation Filter with Channel and Spatial Reliability](https://mlsanity.com/paper/1611.08461/) (2016)
- [Self-Supervised Siamese Learning on Stereo Image Pairs for Depth Estimation in Robotic Surgery](https://mlsanity.com/paper/1705.08260/) (2017)
- [Unsupervised Deep Representation Learning for Real-Time Tracking](https://mlsanity.com/paper/2007.11984/) (2020)
- [Autonomous Tissue Scanning under Free-Form Motion for Intraoperative Tissue Characterisation](https://mlsanity.com/paper/2005.05050/) (2020)
- [Caveats on the first-generation da Vinci Research Kit: latent technical constraints and essential calibrations](https://mlsanity.com/paper/2210.13598/) (2022)
- [Regularising disparity estimation via multi task learning with structured light reconstruction](https://mlsanity.com/paper/2301.08140/) (2023)

## Built on by (most cited first, 14 of 14)

- [Tracking and Mapping in Medical Computer Vision: A Review](https://mlsanity.com/paper/2310.11475/) (2023)
- [Surgical Tattoos in Infrared: A Dataset for Quantifying Tissue Tracking and Mapping](https://mlsanity.com/paper/2309.16782/) (2023)
- [Online 3D reconstruction and dense tracking in endoscopic videos](https://mlsanity.com/paper/2409.06037/) (2024)
- [Point Tracking in Surgery--The 2024 Surgical Tattoos in Infrared (STIR) Challenge](https://mlsanity.com/paper/2503.24306/) (2025)
- [Tracking Everything in Robotic-Assisted Surgery](https://mlsanity.com/paper/2409.19821/) (2024)
- [Generalized Recognition of Basic Surgical Actions Enables Skill Assessment and Vision-Language-Model-based Surgical Planning](https://mlsanity.com/paper/2603.12787/) (2026)
- [SurgSigma: A Spectrum of Large-Scale Multimodal Data and Foundation Models for Surgical Intelligence](https://mlsanity.com/paper/2603.16822/) (2026)
- [Endo-TTAP: Robust Endoscopic Tissue Tracking via Multi-Facet Guided Attention and Hybrid Flow-point Supervision](https://mlsanity.com/paper/2503.22394/) (2025)
- [SurgiATM: A Physics-Guided Plug-and-Play Model for Deep Learning-Based Smoke Removal in Laparoscopic Surgery](https://mlsanity.com/paper/2511.05059/) (2025)
- [Bridging Vision and Language for Robust Context-Aware Surgical Point Tracking: The VL-SurgPT Dataset and Benchmark](https://mlsanity.com/paper/2511.12026/) (2025)
- [Track2Map: Online Deformable SLAM with Motion-Aware Pose Optimization in Robotic Surgery](https://mlsanity.com/paper/2607.08408/) (2026)
- [Point Tracking in Surgery--The 2025 Surgical Tattoos in Infrared Challenge (STIRC2025)](https://mlsanity.com/paper/2607.12939/) (2026)
- [Stitch-Inferencer: Enhance Endoscopic Video Segmentation and Tracking via Panoramic Reconstruction](https://mlsanity.com/paper/2607.14968/) (2026)
- [S3-Tracker: Self-Supervised Surgical Tissue Tracking With Contrastive Random Walks](https://mlsanity.com/paper/2609.14313/) (2026)

## Abstract (arXiv)

This paper introduces the ``SurgT: Surgical Tracking" challenge which was organised in conjunction with MICCAI 2022. There were two purposes for the creation of this challenge: (1) the establishment of the first standardised benchmark for the research community to assess soft-tissue trackers; and (2) to encourage the development of unsupervised deep learning methods, given the lack of annotated data in surgery. A dataset of 157 stereo endoscopic videos from 20 clinical cases, along with stereo camera calibration parameters, have been provided. Participants were assigned the task of developing algorithms to track the movement of soft tissues, represented by bounding boxes, in stereo endoscopic videos. At the end of the challenge, the developed methods were assessed on a previously hidden test subset. This assessment uses benchmarking metrics that were purposely developed for this challenge, to verify the efficacy of unsupervised deep learning algorithms in tracking soft-tissue. The metric used for ranking the methods was the Expected Average Overlap (EAO) score, which measures the average overlap between a tracker's and the ground truth bounding boxes. Coming first in the challenge was the deep learning submission by ICVS-2Ai with a superior EAO score of 0.617. This method employs ARFlow to estimate unsupervised dense optical flow from cropped images, using photometric and regularization losses. Second, Jmees with an EAO of 0.583, uses deep learning for surgical tool segmentation on top of a non-deep learning baseline method: CSRT. CSRT by itself scores a similar EAO of 0.563. The results from this challenge show that currently, non-deep learning methods are still competitive. The dataset and benchmarking tool created for this challenge have been made publicly available at https://surgt.grand-challenge.org/.