# Hyper-Transformer for Amodal Completion

arXiv: https://arxiv.org/abs/2405.19949 · HTML: https://mlsanity.com/paper/2405.19949/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 37 references that are in the map.
- First posted on arXiv 30 May 2024.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [HyperNetworks](https://mlsanity.com/paper/1609.09106/) (2016)
- [Segment Anything in High Quality](https://mlsanity.com/paper/2306.01567/) (2023)
- [Dynamic Convolution: Attention over Convolution Kernels](https://mlsanity.com/paper/1912.03458/) (2019)
- [Fast Segment Anything](https://mlsanity.com/paper/2306.12156/) (2023)
- [HyperStyle: StyleGAN Inversion with HyperNetworks for Real Image Editing](https://mlsanity.com/paper/2111.15666/) (2021)
- [Deep Occlusion-Aware Instance Segmentation with Overlapping BiLayers](https://mlsanity.com/paper/2103.12340/) (2021)
- [Semantic Amodal Segmentation](https://mlsanity.com/paper/1509.01329/) (2015)
- [Self-Supervised Scene De-occlusion](https://mlsanity.com/paper/2004.02788/) (2020)
- [HyperSeg: Patch-wise Hypernetwork for Real-time Semantic Segmentation](https://mlsanity.com/paper/2012.11582/) (2020)
- [Learning to See the Invisible: End-to-End Trainable Amodal Instance Segmentation](https://mlsanity.com/paper/1804.08864/) (2018)
- [SeGAN: Segmenting and Generating the Invisible](https://mlsanity.com/paper/1703.10239/) (2017)
- [Amodal Segmentation Based on Visible Region Segmentation and Shape Prior](https://mlsanity.com/paper/2012.05598/) (2020)
- [Unseen Object Amodal Instance Segmentation via Hierarchical Occlusion Modeling](https://mlsanity.com/paper/2109.11103/) (2021)
- [AISFormer: Amodal Instance Segmentation with Transformer](https://mlsanity.com/paper/2210.06323/) (2022)
- [Occlusion-Aware Video Object Inpainting](https://mlsanity.com/paper/2108.06765/) (2021)
- [Amodal Segmentation through Out-of-Task and Out-of-Distribution Generalization with a Bayesian Model](https://mlsanity.com/paper/2010.13175/) (2020)
- [Unsupervised Object Learning via Common Fate](https://mlsanity.com/paper/2110.06562/) (2021)
- [Coarse-to-Fine Amodal Segmentation with Shape Prior](https://mlsanity.com/paper/2308.16825/) (2023)
- [MVTec D2S: Densely Segmented Supermarket Dataset](https://mlsanity.com/paper/1804.08292/) (2018)
- [Learning 6-DoF Object Poses to Grasp Category-level Objects by Language Instructions](https://mlsanity.com/paper/2205.04028/) (2022)
- [Self-supervised Amodal Video Object Segmentation](https://mlsanity.com/paper/2210.12733/) (2022)
- [CVAE-H: Conditionalizing Variational Autoencoders via Hypernetworks and Trajectory Forecasting for Autonomous Driving](https://mlsanity.com/paper/2201.09874/) (2022)
- [Overparametrization of HyperNetworks at Fixed FLOP-Count Enables Fast Neural Image Enhancement](https://mlsanity.com/paper/2105.08470/) (2021)
- [ImpDet: Exploring Implicit Fields for 3D Object Detection](https://mlsanity.com/paper/2203.17240/) (2022)

## Built on by (most cited first, 1 of 1)

- [LAC-Net: Linear-Fusion Attention-Guided Convolutional Network for Accurate Robotic Grasping Under the Occlusion](https://mlsanity.com/paper/2408.03238/) (2024)

## Abstract (arXiv)

Amodal object completion is a complex task that involves predicting the invisible parts of an object based on visible segments and background information. Learning shape priors is crucial for effective amodal completion, but traditional methods often rely on two-stage processes or additional information, leading to inefficiencies and potential error accumulation. To address these shortcomings, we introduce a novel framework named the Hyper-Transformer Amodal Network (H-TAN). This framework utilizes a hyper transformer equipped with a dynamic convolution head to directly learn shape priors and accurately predict amodal masks. Specifically, H-TAN uses a dual-branch structure to extract multi-scale features from both images and masks. The multi-scale features from the image branch guide the hyper transformer in learning shape priors and in generating the weights for dynamic convolution tailored to each instance. The dynamic convolution head then uses the features from the mask branch to predict precise amodal masks. We extensively evaluate our model on three benchmark datasets: KINS, COCOA-cls, and D2SA, where H-TAN demonstrated superior performance compared to existing methods. Additional experiments validate the effectiveness and stability of the novel hyper transformer in our framework.