# Click Carving: Segmenting Objects in Video with Point Clicks

arXiv: https://arxiv.org/abs/1607.01115 · HTML: https://mlsanity.com/paper/1607.01115/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jul 2016.

## Built on by (most cited first, 8 of 8)

- [Full-Duplex Strategy for Video Object Segmentation](https://mlsanity.com/paper/2108.03151/) (2021)
- [Motion-aware Memory Network for Fast Video Salient Object Detection](https://mlsanity.com/paper/2208.00946/) (2022)
- [Scaling up instance annotation via label propagation](https://mlsanity.com/paper/2110.02277/) (2021)
- [Revisiting Click-based Interactive Video Object Segmentation](https://mlsanity.com/paper/2203.01784/) (2022)
- [Strike the Balance: On-the-Fly Uncertainty based User Interactions for Long-Term Video Object Segmentation](https://mlsanity.com/paper/2408.00169/) (2024)
- [Evaluating and Improving Interactions with Hazy Oracles](https://mlsanity.com/paper/2110.10206/) (2021)
- [IDPro: Flexible Interactive Video Object Segmentation by ID-queried Concurrent Propagation](https://mlsanity.com/paper/2401.12480/) (2024)
- [Autonomous Curiosity for Real-Time Training Onboard Robotic Agents](https://mlsanity.com/paper/2109.00927/) (2021)

## Abstract (arXiv)

We present a novel form of interactive video object segmentation where a few clicks by the user helps the system produce a full spatio-temporal segmentation of the object of interest. Whereas conventional interactive pipelines take the user's initialization as a starting point, we show the value in the system taking the lead even in initialization. In particular, for a given video frame, the system precomputes a ranked list of thousands of possible segmentation hypotheses (also referred to as object region proposals) using image and motion cues. Then, the user looks at the top ranked proposals, and clicks on the object boundary to carve away erroneous ones. This process iterates (typically 2-3 times), and each time the system revises the top ranked proposal set, until the user is satisfied with a resulting segmentation mask. Finally, the mask is propagated across the video to produce a spatio-temporal object tube. On three challenging datasets, we provide extensive comparisons with both existing work and simpler alternative methods. In all, the proposed Click Carving approach strikes an excellent balance of accuracy and human effort. It outperforms all similarly fast methods, and is competitive or better than those requiring 2 to 12 times the effort.