mlsanityData to 14 Oct 2026

Occlusion-Aware Video Object Inpainting

arXiv 2108.06765 ↗First posted 15 Aug 2021cs.CVcited by 30

Not on the map. Why?

Abstract (arXiv)

Conventional video inpainting is neither object-oriented nor occlusion-aware, making it liable to obvious artifacts when large occluded object regions are inpainted. This paper presents occlusion-aware video object inpainting, which recovers both the complete shape and appearance for occluded objects in videos given their visible mask segmentation. To facilitate this new research, we construct the first large-scale video object inpainting benchmark YouTube-VOI to provide realistic occlusion scenarios with both occluded and visible object masks available. Our technical contribution VOIN jointly performs video object shape completion and occluded texture generation. In particular, the shape completion module models long-range object coherence while the flow completion module recovers accurate flow with sharp motion boundary, for propagating temporally-consistent texture to the same moving object across frames. For more realistic results, VOIN is optimized using both T-PatchGAN and a new spatio-temporal attention-based multi-class discriminator. Finally, we compare VOIN and strong baselines on YouTube-VOI. Experimental results clearly demonstrate the efficacy of our method including inpainting complex and dynamic objects. VOIN degrades gracefully with inaccurate input visible mask.

Key facts. Cited by 30 papers in the map (data through 14 Oct 2026). Builds on 43 of its 73 references that are in the map. First posted on arXiv 15 Aug 2021.

Builds on

43 of 73 references.

References · 43

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  5. 2020End-to-End Object Detection with Transformerscited by 7,414
  6. 2017Squeeze-and-Excitation Networkscited by 4,789
  7. 2018CBAM: Convolutional Block Attention Modulecited by 2,477
  8. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
  9. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,292
  10. 2018TSM: Temporal Shift Module for Efficient Video Understandingcited by 856
  11. 2018Free-Form Image Inpainting with Gated Convolutioncited by 594
  12. 2018Generative Image Inpainting with Contextual Attentioncited by 580
31 more
  1. 2020End-to-End Video Instance Segmentation with Transformerscited by 443
  2. 2019Video Instance Segmentationcited by 393
  3. 2018YouTube-VOS: A Large-Scale Video Object Segmentation Benchmarkcited by 383
  4. 2020Spatio-Temporal Graph Transformer Networks for Pedestrian Trajectory Predictioncited by 218
  5. 2020Learning Joint Spatial-Temporal Transformations for Video Inpaintingcited by 163
  6. 2017Optimizing the Latent Space of Generative Networkscited by 157
  7. 2019Deep Flow-Guided Video Inpaintingcited by 116
  8. 2019Deep Video Inpaintingcited by 109
  9. 2020SipMask: Spatial Information Preservation for Fast Image and Video Instance Segmentationcited by 108
  10. 2020Flow-edge Guided Video Completioncited by 104
  11. 2021Deep Occlusion-Aware Instance Segmentation with Overlapping BiLayerscited by 104
  12. 2015Semantic Amodal Segmentationcited by 103
  13. 2020Self-Supervised Scene De-occlusioncited by 96
  14. 2019RVOS: End-to-End Recurrent Network for Video Object Segmentationcited by 88
  15. 2019Free-form Video Inpainting with 3D Gated Convolution and Temporal PatchGANcited by 87
  16. 2020Learning Fast and Robust Target Models for Video Object Segmentationcited by 76
  17. 2016Amodal Instance Segmentationcited by 63
  18. 2017Repulsion Loss: Detecting Pedestrians in a Crowdcited by 62
  19. 2018Learning to See the Invisible: End-to-End Trainable Amodal Instance Segmentationcited by 62
  20. 2017SeGAN: Segmenting and Generating the Invisiblecited by 60
  21. 2018Video Inpainting by Jointly Learning Temporal Structure and Spatial Detailscited by 58
  22. 2019Copy-and-Paste Networks for Deep Video Inpaintingcited by 58
  23. 2020Robust Object Detection under Occlusion with Context-Aware CompositionalNetscited by 54
  24. 2020Compositional Convolutional Neural Networks: A Deep Architecture with Innate Robustness to Partial Occlusioncited by 47
  25. 2019Onion-Peel Networks for Deep Video Completioncited by 46
  26. 2015Video Inpainting of Complex Scenescited by 41
  27. 2019An Internal Learning Approach to Video Inpaintingcited by 33
  28. 2020GSNet: Joint Vehicle Pose and Shape Reconstruction with Geometrical and Scene-aware Supervisioncited by 28
  29. 2019Visualizing the Invisible: Occluded Vehicle Segmentation and Recoverycited by 24
  30. 2015Amodal Completion and Size Constancy in Natural Scenescited by 21
  31. 2014Parsing Occluded People by Flexible Compositionscited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 5; 2023: 5; 2024: 10; 2025: 5; 2026: 4.

30 papers.

Sort:

By area:

  1. 2023ProPainter: Improving Propagation and Transformer for Video Inpaintingcited by 197
  2. 2021Mask Transfiner for High-Quality Instance Segmentationcited by 64
  3. 2024DreamScene4D: Dynamic Multi-Object Scene Generation from Monocular Videoscited by 63
  4. 2022Flow-Guided Transformer for Video Inpaintingcited by 59
  5. 2024Deep Learning-based Image and Video Inpainting: A Surveycited by 44
  6. 2024Generative Video Propagationcited by 36
  7. 2023Robustness of SAM: Segment Anything Under Corruptions and Beyondcited by 31
  8. 2023Coarse-to-Fine Amodal Segmentation with Shape Priorcited by 29
  9. 2022Video Mask Transfiner for High-Quality Video Instance Segmentationcited by 28
  10. 2022Self-supervised Amodal Video Object Segmentationcited by 14
  11. 2025TACO: Taming Diffusion for in-the-wild Video Amodal Completioncited by 10
  12. 2022Shape-guided Object Inpaintingcited by 8
18 more
  1. 2025Bridging Perspectives: A Survey on Cross-view Collaborative Intelligence with Egocentric-Exocentric Visioncited by 8
  2. 2024LiVOS: Light Video Object Segmentation with Gated Linear Matchingcited by 7
  3. 2023Exploiting Optical Flow Guidance for Transformer-Based Video Inpaintingcited by 6
  4. 2022Occlusion-Aware Instance Segmentation via BiLayer Network Architecturescited by 5
  5. 2023Layout-guided Indoor Panorama Inpainting with Plane-aware Normalizationcited by 5
  6. 2026From Ideal to Real: Stable Video Object Removal under Imperfect Conditionscited by 5
  7. 2024Deep video representation learning: a surveycited by 4
  8. 2024Towards Online Real-Time Memory-based Video Inpainting Transformerscited by 3
  9. 2025Removing Cost Volumes from Optical Flow Estimatorscited by 3
  10. 2024LAC-Net: Linear-Fusion Attention-Guided Convolutional Network for Accurate Robotic Grasping Under the Occlusioncited by 2
  11. 2024Hyper-Transformer for Amodal Completioncited by 1
  12. 2024Reimagining Reality: A Comprehensive Survey of Video Inpainting Techniquescited by 0
  13. 2024Optimised ProPainter for Video Diminished Reality Inpaintingcited by 0
  14. 2025MagicRoad: Semantic-Aware 3D Road Surface Reconstruction via Obstacle Inpaintingcited by 0
  15. 2025Object-Aware 4D Human Motion Generationcited by 0
  16. 2026Transformer-Based Inpainting for Real-Time 3D Streaming in Sparse Multi-Camera Setupscited by 0
  17. 2026Occlusion-Aware Physics-Semantic Keyframe Selection for Robust Video Editingcited by 0
  18. 2026WiFlow: Estimating Optical Flow using WiFi Channel State Informationcited by 0