# Memory Aggregation Networks for Efficient Interactive Video Object Segmentation

arXiv: https://arxiv.org/abs/2003.13246 · HTML: https://mlsanity.com/paper/2003.13246/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Mar 2020.

## Built on by (most cited first, 30 of 40)

- [MOSE: A New Dataset for Video Object Segmentation in Complex Scenes](https://mlsanity.com/paper/2302.01872/) (2023)
- [Associating Objects with Transformers for Video Object Segmentation](https://mlsanity.com/paper/2106.02638/) (2021)
- [A Survey on Deep Learning Technique for Video Segmentation](https://mlsanity.com/paper/2107.01153/) (2021)
- [Modular Interactive Video Object Segmentation: Interaction-to-Mask, Propagation and Difference-Aware Fusion](https://mlsanity.com/paper/2103.07941/) (2021)
- [Segment Anything Meets Point Tracking](https://mlsanity.com/paper/2307.01197/) (2023)
- [Full-Duplex Strategy for Video Object Segmentation](https://mlsanity.com/paper/2108.03151/) (2021)
- [MOSEv2: A More Challenging Dataset for Video Object Segmentation in Complex Scenes](https://mlsanity.com/paper/2508.05630/) (2025)
- [Watching You: Global-guided Reciprocal Learning for Video-based Person Re-identification](https://mlsanity.com/paper/2103.04337/) (2021)
- [LVOS: A Benchmark for Large-scale Long-term Video Object Segmentation](https://mlsanity.com/paper/2404.19326/) (2024)
- [ClothFormer:Taming Video Virtual Try-on in All Module](https://mlsanity.com/paper/2204.12151/) (2022)
- [MAMo: Leveraging Memory and Attention for Monocular Video Depth Estimation](https://mlsanity.com/paper/2307.14336/) (2023)
- [MV-TON: Memory-based Video Virtual Try-on network](https://mlsanity.com/paper/2108.07502/) (2021)
- [Memory-guided Unsupervised Image-to-image Translation](https://mlsanity.com/paper/2104.05170/) (2021)
- [Guided Interactive Video Object Segmentation Using Reliability-Based Attention Maps](https://mlsanity.com/paper/2104.10386/) (2021)
- [Boosting Video Object Segmentation via Space-time Correspondence Learning](https://mlsanity.com/paper/2304.06211/) (2023)
- [End-to-End Modeling via Information Tree for One-Shot Natural Language Spatial Video Grounding](https://mlsanity.com/paper/2203.08013/) (2022)
- [MarS3D: A Plug-and-Play Motion-Aware Model for Semantic Segmentation on Multi-Scan 3D Point Clouds](https://mlsanity.com/paper/2307.09316/) (2023)
- [Learning to Recommend Frame for Interactive Video Object Segmentation in the Wild](https://mlsanity.com/paper/2103.10391/) (2021)
- [Unified Mask Embedding and Correspondence Learning for Self-Supervised Video Segmentation](https://mlsanity.com/paper/2303.10100/) (2023)
- [UVOSAM: A Mask-free Paradigm for Unsupervised Video Object Segmentation via Segment Anything Model](https://mlsanity.com/paper/2305.12659/) (2023)
- [HumanDiffusion: a Coarse-to-Fine Alignment Diffusion Framework for Controllable Text-Driven Person Image Generation](https://mlsanity.com/paper/2211.06235/) (2022)
- [VideoClick: Video Object Segmentation with a Single Click](https://mlsanity.com/paper/2101.06545/) (2021)
- [In-N-Out Generative Learning for Dense Unsupervised Video Segmentation](https://mlsanity.com/paper/2203.15312/) (2022)
- [LiVOS: Light Video Object Segmentation with Gated Linear Matching](https://mlsanity.com/paper/2411.02818/) (2024)
- [General and Task-Oriented Video Segmentation](https://mlsanity.com/paper/2407.06540/) (2024)
- [Revisiting Click-based Interactive Video Object Segmentation](https://mlsanity.com/paper/2203.01784/) (2022)
- [Fast Interactive Video Object Segmentation with Graph Neural Networks](https://mlsanity.com/paper/2103.03821/) (2021)
- [Strike the Balance: On-the-Fly Uncertainty based User Interactions for Long-Term Video Object Segmentation](https://mlsanity.com/paper/2408.00169/) (2024)
- [Generalizing Interactive Backpropagating Refinement for Dense Prediction](https://mlsanity.com/paper/2112.10969/) (2021)
- [DiDA: Video Object Segmentation with Distillation Learning of Deformable Attention](https://mlsanity.com/paper/2401.13937/) (2024)

## Abstract (arXiv)

Interactive video object segmentation (iVOS) aims at efficiently harvesting high-quality segmentation masks of the target object in a video with user interactions. Most previous state-of-the-arts tackle the iVOS with two independent networks for conducting user interaction and temporal propagation, respectively, leading to inefficiencies during the inference stage. In this work, we propose a unified framework, named Memory Aggregation Networks (MA-Net), to address the challenging iVOS in a more efficient way. Our MA-Net integrates the interaction and the propagation operations into a single network, which significantly promotes the efficiency of iVOS in the scheme of multi-round interactions. More importantly, we propose a simple yet effective memory aggregation mechanism to record the informative knowledge from the previous interaction rounds, improving the robustness in discovering challenging objects of interest greatly. We conduct extensive experiments on the validation set of DAVIS Challenge 2018 benchmark. In particular, our MA-Net achieves the J@60 score of 76.1% without any bells and whistles, outperforming the state-of-the-arts with more than 2.7%.