# Iterative multi-path tracking for video and volume segmentation with sparse point supervision

arXiv: https://arxiv.org/abs/1809.00970 · HTML: https://mlsanity.com/paper/1809.00970/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Aug 2018.

## Built on by (most cited first, 6 of 6)

- [Medical Image Segmentation with Limited Supervision: A Review of Deep Network Models](https://mlsanity.com/paper/2103.00429/) (2021)
- [Surgical Scene Understanding in the Era of Foundation AI Models: A Comprehensive Review](https://mlsanity.com/paper/2502.14886/) (2025)
- [Reducing Annotating Load: Active Learning with Synthetic Images in Surgical Instrument Segmentation](https://mlsanity.com/paper/2108.03534/) (2021)
- [A comprehensive survey on recent deep learning-based methods applied to surgical data](https://mlsanity.com/paper/2209.01435/) (2022)
- [A Positive/Unlabeled Approach for the Segmentation of Medical Sequences using Point-Wise Supervision](https://mlsanity.com/paper/2107.08394/) (2021)
- [Partial annotations for the segmentation of large structures with low annotation cost](https://mlsanity.com/paper/2209.12216/) (2022)

## Abstract (arXiv)

Recent machine learning strategies for segmentation tasks have shown great ability when trained on large pixel-wise annotated image datasets. It remains a major challenge however to aggregate such datasets, as the time and monetary cost associated with collecting extensive annotations is extremely high. This is particularly the case for generating precise pixel-wise annotations in video and volumetric image data. To this end, this work presents a novel framework to produce pixel-wise segmentations using minimal supervision. Our method relies on 2D point supervision, whereby a single 2D location within an object of interest is provided on each image of the data. Our method then estimates the object appearance in a semi-supervised fashion by learning object-image-specific features and by using these in a semi-supervised learning framework. Our object model is then used in a graph-based optimization problem that takes into account all provided locations and the image data in order to infer the complete pixel-wise segmentation. In practice, we solve this optimally as a tracking problem using a K-shortest path approach. Both the object model and segmentation are then refined iteratively to further improve the final segmentation. We show that by collecting 2D locations using a gaze tracker, our approach can provide state-of-the-art segmentations on a range of objects and image modalities (video and 3D volumes), and that these can then be used to train supervised machine learning classifiers.