mlsanityData to 7 Oct 2026

Learning Joint Spatial-Temporal Transformations for Video Inpainting

arXiv 2007.10247 ↗First posted 20 Jul 2020cs.CVcited by 162

Not on the map. Why?

Abstract (arXiv)

High-quality video inpainting that completes missing regions in video frames is a promising yet challenging task. State-of-the-art approaches adopt attention models to complete a frame by searching missing contents from reference frames, and further complete whole videos frame by frame. However, these approaches can suffer from inconsistent attention results along spatial and temporal dimensions, which often leads to blurriness and temporal artifacts in videos. In this paper, we propose to learn a joint Spatial-Temporal Transformer Network (STTN) for video inpainting. Specifically, we simultaneously fill missing regions in all input frames by self-attention, and propose to optimize STTN by a spatial-temporal adversarial loss. To show the superiority of the proposed model, we conduct both quantitative and qualitative evaluations by using standard stationary masks and more realistic moving object masks. Demo videos are available at https://github.com/researchmm/STTN.

Key facts. Cited by 162 papers in the map (data through 7 Oct 2026). First posted on arXiv 20 Jul 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 45; 2022: 36; 2023: 33; 2024: 18; 2025: 18; 2026: 12.

162 papers.

Sort:

By area:

  1. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,096
  2. 2021CvT: Introducing Convolutions to Vision Transformerscited by 946
  3. 2021Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modelingcited by 647
  4. 2021Twins: Revisiting the Design of Spatial Attention in Vision Transformerscited by 565
  5. 2021TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classificationcited by 549
  6. 2022FlowFormer: A Transformer Architecture for Optical Flowcited by 273
  7. 2021TransVG: End-to-End Visual Grounding with Transformerscited by 269
  8. 2021Incorporating Convolution Designs into Visual Transformerscited by 248
  9. 2021Multi-Scale Vision Longformer: A New Vision Transformer for High-Resolution Image Encodingcited by 207
  10. 2023ProPainter: Improving Propagation and Transformer for Video Inpaintingcited by 193
  11. 2021Chasing Sparsity in Vision Transformers: An End-to-End Explorationcited by 181
  12. 2021TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Upcited by 162
38 more
  1. 2023Swap Attention in Spatiotemporal Diffusions for Text-to-Video Generationcited by 153
  2. 2021All Tokens Matter: Token Labeling for Training Better Vision Transformerscited by 144
  3. 2022Stripformer: Strip Transformer for Fast Image Deblurringcited by 139
  4. 2021PhysFormer: Facial Video-based Physiological Measurement with Temporal Difference Transformercited by 118
  5. 2023FlowFormer++: Masked Cost Volume Autoencoding for Pretraining Optical Flow Estimationcited by 114
  6. 2022Towards An End-to-End Framework for Flow-Guided Video Inpaintingcited by 111
  7. 2021FuseFormer: Fusing Fine-Grained Information in Transformers for Video Inpaintingcited by 94
  8. 2021Aggregated Contextual Transformations for High-Resolution Image Inpaintingcited by 89
  9. 2025VideoPainter: Any-length Video Inpainting and Editing with Plug-and-Play Context Controlcited by 83
  10. 2023AVID: Any-Length Video Inpainting with Diffusion Modelcited by 81
  11. 2022Improving Vision Transformers by Revisiting High-frequency Componentscited by 75
  12. 2021M3DeTR: Multi-representation, Multi-scale, Mutual-relation 3D Object Detection with Transformerscited by 74
  13. 2022Representation Compensation Networks for Continual Semantic Segmentationcited by 72
  14. 2022Removing Objects From Neural Radiance Fieldscited by 66
  15. 2022Video Transformers: A Surveycited by 63
  16. 2022Learning Trajectory-Aware Transformer for Video Super-Resolutioncited by 61
  17. 2022Flow-Guided Transformer for Video Inpaintingcited by 59
  18. 2024InFusion: Inpainting 3D Gaussians via Learning Depth Completion from Diffusion Priorcited by 55
  19. 2022TAPE: Task-Agnostic Prior Embedding for Image Restorationcited by 54
  20. 2023PhysFormer++: Facial Video-based Physiological Measurement with SlowFast Temporal Difference Transformercited by 46
  21. 2024Deep Learning-based Image and Video Inpainting: A Surveycited by 44
  22. 2022TransVG++: End-to-End Visual Grounding with Language Conditioned Vision Transformercited by 41
  23. 2021Domain-Aware Universal Style Transfercited by 40
  24. 2021Refiner: Refining Self-attention for Vision Transformerscited by 38
  25. 2022The Principle of Diversity: Training Stronger Vision Transformers Calls for Reducing All Levels of Redundancycited by 35
  26. 2021Progressive Temporal Feature Alignment Network for Video Inpaintingcited by 34
  27. 2021Revitalizing CNN Attentions via Transformers in Self-Supervised Visual Representation Learningcited by 34
  28. 2022Class-Aware Adversarial Transformers for Medical Image Segmentationcited by 33
  29. 2022Low-Light Video Enhancement with Synthetic Event Guidancecited by 33
  30. 2023UMMAFormer: A Universal Multimodal-adaptive Transformer Framework for Temporal Forgery Localizationcited by 33
  31. 2024Towards Language-Driven Video Inpainting via Multimodal Large Language Modelscited by 33
  32. 2025VideoShield: Regulating Diffusion-based Video Generation Models via Watermarkingcited by 32
  33. 2021Transformer-Based Source-Free Domain Adaptationcited by 30
  34. 2021Occlusion-Aware Video Object Inpaintingcited by 30
  35. 2021A Video Is Worth Three Views: Trigeminal Transformers for Video-based Person Re-identificationcited by 28
  36. 2021Decoupled Spatial-Temporal Transformer for Video Inpaintingcited by 28
  37. 2024V2A-Mark: Versatile Deep Visual-Audio Watermarking for Manipulation Localization and Copyright Protectioncited by 27
  38. 2022ClothFormer:Taming Video Virtual Try-on in All Modulecited by 26

50 most cited of 162 All 162 (JSON)