# f-BRS: Rethinking Backpropagating Refinement for Interactive Segmentation

arXiv: https://arxiv.org/abs/2001.10331 · HTML: https://mlsanity.com/paper/2001.10331/

## Key facts

- Cited by 117 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Jan 2020.

## Built on by (most cited first, 30 of 117)

- [BEHAVE: Dataset and Method for Tracking Human Object Interactions](https://mlsanity.com/paper/2204.06950/) (2022)
- [Tracking Anything with Decoupled Video Segmentation](https://mlsanity.com/paper/2309.03903/) (2023)
- [Reviving Iterative Training with Mask Guidance for Interactive Segmentation](https://mlsanity.com/paper/2102.06583/) (2021)
- [COUCH: Towards Controllable Human-Chair Interactions](https://mlsanity.com/paper/2205.00541/) (2022)
- [Modular Interactive Video Object Segmentation: Interaction-to-Mask, Propagation and Difference-Aware Fusion](https://mlsanity.com/paper/2103.07941/) (2021)
- [SimpleClick: Interactive Image Segmentation with Simple Vision Transformers](https://mlsanity.com/paper/2210.11006/) (2022)
- [FocalClick: Towards Practical Interactive Image Segmentation](https://mlsanity.com/paper/2204.02574/) (2022)
- [Caption Anything: Interactive Image Description with Diverse Multimodal Controls](https://mlsanity.com/paper/2305.02677/) (2023)
- [Interactive Segmentation of Radiance Fields](https://mlsanity.com/paper/2212.13545/) (2022)
- [Visibility Aware Human-Object Interaction Tracking from Single RGB Camera](https://mlsanity.com/paper/2303.16479/) (2023)
- [ScribblePrompt: Fast and Flexible Interactive Segmentation for Any Biomedical Image](https://mlsanity.com/paper/2312.07381/) (2023)
- [PseudoClick: Interactive Image Segmentation with Click Imitation](https://mlsanity.com/paper/2207.05282/) (2022)
- [EdgeFlow: Achieving Practical Interactive Segmentation with Edge-Guided Flow](https://mlsanity.com/paper/2109.09406/) (2021)
- [Template Free Reconstruction of Human-object Interaction with Procedural Interaction Generation](https://mlsanity.com/paper/2312.07063/) (2023)
- [Shape and Material Capture at Home](https://mlsanity.com/paper/2104.06397/) (2021)
- [Unsupervised Video Object Segmentation via Prototype Memory Network](https://mlsanity.com/paper/2209.03712/) (2022)
- [Interaction Replica: Tracking Human-Object Interaction and Scene Changes From Human Motion](https://mlsanity.com/paper/2205.02830/) (2022)
- [Deep Interactive Segmentation of Medical Images: A Systematic Review and Taxonomy](https://mlsanity.com/paper/2311.13964/) (2023)
- [CloSe: A 3D Clothing Segmentation Dataset and Model](https://mlsanity.com/paper/2401.12051/) (2024)
- [InterFormer: Real-time Interactive Image Segmentation](https://mlsanity.com/paper/2304.02942/) (2023)
- [CFR-ICL: Cascade-Forward Refinement with Iterative Click Loss for Interactive Image Segmentation](https://mlsanity.com/paper/2303.05620/) (2023)
- [Focused and Collaborative Feedback Integration for Interactive Image Segmentation](https://mlsanity.com/paper/2303.11880/) (2023)
- [Measured Albedo in the Wild: Filling the Gap in Intrinsics Evaluation](https://mlsanity.com/paper/2306.15662/) (2023)
- [Visual Position Prompt for MLLM based Visual Grounding](https://mlsanity.com/paper/2503.15426/) (2025)
- [Interactive Multi-Class Tiny-Object Detection](https://mlsanity.com/paper/2203.15266/) (2022)
- [Interactive Segmentation as Gaussian Process Classification](https://mlsanity.com/paper/2302.14578/) (2023)
- [DynaMITe: Dynamic Query Bootstrapping for Multi-object Interactive Segmentation Transformer](https://mlsanity.com/paper/2304.06668/) (2023)
- [CARI4D: Category Agnostic 4D Reconstruction of Human-Object Interaction](https://mlsanity.com/paper/2512.11988/) (2025)
- [Boundary-aware Supervoxel-level Iteratively Refined Interactive 3D Image Segmentation with Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2303.10692/) (2023)
- [R-MAE: Regions Meet Masked Autoencoders](https://mlsanity.com/paper/2306.05411/) (2023)

## Abstract (arXiv)

Deep neural networks have become a mainstream approach to interactive segmentation. As we show in our experiments, while for some images a trained network provides accurate segmentation result with just a few clicks, for some unknown objects it cannot achieve satisfactory result even with a large amount of user input. Recently proposed backpropagating refinement (BRS) scheme introduces an optimization problem for interactive segmentation that results in significantly better performance for the hard cases. At the same time, BRS requires running forward and backward pass through a deep network several times that leads to significantly increased computational budget per click compared to other methods. We propose f-BRS (feature backpropagating refinement scheme) that solves an optimization problem with respect to auxiliary variables instead of the network inputs, and requires running forward and backward pass just for a small part of a network. Experiments on GrabCut, Berkeley, DAVIS and SBD datasets set new state-of-the-art at an order of magnitude lower time per click compared to original BRS. The code and trained models are available at https://github.com/saic-vul/fbrs_interactive_segmentation .