# Rethinking Amodal Video Segmentation from Learning Supervised Signals with Object-centric Representation

arXiv: https://arxiv.org/abs/2309.13248 · HTML: https://mlsanity.com/paper/2309.13248/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 38 references that are in the map.
- First posted on arXiv 23 Sep 2023.

## Builds on (most cited first, 29 of 29)

- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning](https://mlsanity.com/paper/1612.06890/) (2016)
- [BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers](https://mlsanity.com/paper/2203.17270/) (2022)
- [Object-Centric Learning with Slot Attention](https://mlsanity.com/paper/2006.15055/) (2020)
- [Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Items](https://mlsanity.com/paper/2204.11918/) (2022)
- [MONet: Unsupervised Scene Decomposition and Representation](https://mlsanity.com/paper/1901.11390/) (2019)
- [Kubric: A scalable dataset generator](https://mlsanity.com/paper/2203.03570/) (2022)
- [Universal Instance Perception as Object Discovery and Retrieval](https://mlsanity.com/paper/2303.06674/) (2023)
- [Predicting Semantic Map Representations from Images using Pyramid Occupancy Networks](https://mlsanity.com/paper/2003.13402/) (2020)
- [Cross-view Semantic Segmentation for Sensing Surroundings](https://mlsanity.com/paper/1906.03560/) (2019)
- [Instances as Queries](https://mlsanity.com/paper/2105.01928/) (2021)
- [Translating Images into Maps](https://mlsanity.com/paper/2110.00966/) (2021)
- [Deep Occlusion-Aware Instance Segmentation with Overlapping BiLayers](https://mlsanity.com/paper/2103.12340/) (2021)
- [Self-Supervised Scene De-occlusion](https://mlsanity.com/paper/2004.02788/) (2020)
- [Object Scene Representation Transformer](https://mlsanity.com/paper/2206.06922/) (2022)
- [Monocular Semantic Occupancy Grid Mapping with Convolutional Variational Encoder-Decoder Networks](https://mlsanity.com/paper/1804.02176/) (2018)
- [SOLQ: Segmenting Objects by Learning Queries](https://mlsanity.com/paper/2106.02351/) (2021)
- [ISTR: End-to-End Instance Segmentation with Transformers](https://mlsanity.com/paper/2105.00637/) (2021)
- [Amodal Segmentation Based on Visible Region Segmentation and Shape Prior](https://mlsanity.com/paper/2012.05598/) (2020)
- [Restricted Deformable Convolution based Road Scene Semantic Segmentation Using Surround View Cameras](https://mlsanity.com/paper/1801.00708/) (2018)
- [Learning to Look around Objects for Top-View Representations of Outdoor Scenes](https://mlsanity.com/paper/1803.10870/) (2018)
- [Segment as Points for Efficient Online Multi-Object Tracking and Segmentation](https://mlsanity.com/paper/2007.01550/) (2020)
- [AISFormer: Amodal Instance Segmentation with Transformer](https://mlsanity.com/paper/2210.06323/) (2022)
- [Amodal Segmentation through Out-of-Task and Out-of-Distribution Generalization with a Bayesian Model](https://mlsanity.com/paper/2010.13175/) (2020)
- [Self-supervised Amodal Video Object Segmentation](https://mlsanity.com/paper/2210.12733/) (2022)
- [Learning Vector Quantized Shape Code for Amodal Blastomere Instance Segmentation](https://mlsanity.com/paper/2012.00985/) (2020)
- [Efficient Semantic Segmentation for Visual Bird's-eye View Interpretation](https://mlsanity.com/paper/1811.12008/) (2018)

## Built on by (most cited first, 9 of 9)

- [Adaptive Slot Attention: Object Discovery with Dynamic Slot Number](https://mlsanity.com/paper/2406.09196/) (2024)
- [Using Diffusion Priors for Video Amodal Segmentation](https://mlsanity.com/paper/2412.04623/) (2024)
- [TACO: Taming Diffusion for in-the-wild Video Amodal Completion](https://mlsanity.com/paper/2503.12049/) (2025)
- [Occlusion-Aware Seamless Segmentation](https://mlsanity.com/paper/2407.02182/) (2024)
- [Unlocking Constraints: Source-Free Occlusion-Aware Seamless Segmentation](https://mlsanity.com/paper/2506.21198/) (2025)
- [Training for X-Ray Vision: Amodal Segmentation, Amodal Content Completion, and View-Invariant Object Representation from Multi-Camera Video](https://mlsanity.com/paper/2507.00339/) (2025)
- [Mitigating Simplicity Bias in OOD Detection through Object Co-occurrence Analysis](https://mlsanity.com/paper/2605.07821/) (2026)
- [Track Anything Behind Everything: Zero-Shot Amodal Video Object Segmentation](https://mlsanity.com/paper/2411.19210/) (2024)
- [A2VIS: Amodal-Aware Approach to Video Instance Segmentation](https://mlsanity.com/paper/2412.01147/) (2024)

## Abstract (arXiv)

Video amodal segmentation is a particularly challenging task in computer vision, which requires to deduce the full shape of an object from the visible parts of it. Recently, some studies have achieved promising performance by using motion flow to integrate information across frames under a self-supervised setting. However, motion flow has a clear limitation by the two factors of moving cameras and object deformation. This paper presents a rethinking to previous works. We particularly leverage the supervised signals with object-centric representation in textitreal-world scenarios. The underlying idea is the supervision signal of the specific object and the features from different views can mutually benefit the deduction of the full mask in any specific frame. We thus propose an Efficient object-centric Representation amodal Segmentation (EoRaS). Specially, beyond solely relying on supervision signals, we design a translation module to project image features into the Bird's-Eye View (BEV), which introduces 3D information to improve current feature quality. Furthermore, we propose a multi-view fusion layer based temporal module which is equipped with a set of object slots and interacts with features from different views by attention mechanism to fulfill sufficient object representation completion. As a result, the full mask of the object can be decoded from image features updated by object slots. Extensive experiments on both real-world and synthetic benchmarks demonstrate the superiority of our proposed method, achieving state-of-the-art performance. Our code will be released at urlhttps://github.com/kfan21/EoRaS.