# Understanding Annotation Error Propagation and Learning an Adaptive Policy for Expert Intervention in Barrett's Video Segmentation

arXiv: https://arxiv.org/abs/2602.21855 · HTML: https://mlsanity.com/paper/2602.21855/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 17 references that are in the map.
- First posted on arXiv 25 Feb 2026.

## Builds on (most cited first, 9 of 9)

- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [A Closer Look at Spatiotemporal Convolutions for Action Recognition](https://mlsanity.com/paper/1711.11248/) (2017)
- [Consistent Estimators for Learning to Defer to an Expert](https://mlsanity.com/paper/2006.01862/) (2020)
- [Video Polyp Segmentation: A Deep Learning Perspective](https://mlsanity.com/paper/2203.14291/) (2022)
- [Video Object Segmentation and Tracking: A Survey](https://mlsanity.com/paper/1904.09172/) (2019)
- [Learning the What and How of Annotation in Video Object Segmentation](https://mlsanity.com/paper/2311.04414/) (2023)
- [CLOC: Contrastive Learning for Ordinal Classification with Multi-Margin N-pair Loss](https://mlsanity.com/paper/2504.17813/) (2025)
- [L2CU: Learning to Complement Unseen Users](https://mlsanity.com/paper/2601.06119/) (2026)

## Abstract (arXiv)

Accurate annotation of endoscopic videos is essential yet time-consuming, particularly for challenging datasets such as dysplasia in Barrett's esophagus, where the affected regions are irregular and lack clear boundaries. Semi-automatic tools like Segment Anything Model 2 (SAM2) can ease this process by propagating annotations across frames, but small errors often accumulate and reduce accuracy, requiring expert review and correction. To address this, we systematically study how annotation errors propagate across different prompt types, namely masks, boxes, and points, and propose Learning-to-Re-Prompt (L2RP), a cost-aware framework that learns when and where to seek expert input. By tuning a human-cost parameter, our method balances annotation effort and segmentation accuracy. Experiments on a private Barrett's dysplasia dataset and the public SUN-SEG benchmark demonstrate improved temporal consistency and superior performance over baseline strategies.