mlsanityData to 7 Oct 2026

Towards An End-to-End Framework for Flow-Guided Video Inpainting

arXiv 2204.02663 ↗First posted 6 Apr 2022eess.IV cs.CVcited by 111

Not on the map. Why?

Cited by 20 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Optical flow, which captures motion information across frames, is exploited in recent video inpainting methods through propagating pixels along its trajectories. However, the hand-crafted flow-based processes in these methods are applied separately to form the whole inpainting pipeline. Thus, these methods are less efficient and rely heavily on the intermediate results from earlier stages. In this paper, we propose an End-to-End framework for Flow-Guided Video Inpainting (E2FGVI) through elaborately designed three trainable modules, namely, flow completion, feature propagation, and content hallucination modules. The three modules correspond with the three stages of previous flow-based methods but can be jointly optimized, leading to a more efficient and effective inpainting process. Experimental results demonstrate that the proposed method outperforms state-of-the-art methods both qualitatively and quantitatively and shows promising efficiency. The code is available at https://github.com/MCG-NKU/E2FGVI.

Key facts. Cited by 111 papers in the map (data through 7 Oct 2026). Builds on 55 of its 68 references that are in the map. First posted on arXiv 6 Apr 2022.

Builds on

55 of 68 references.

References · 55

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  3. 2020End-to-End Object Detection with Transformerscited by 7,384
  4. 2016Layer Normalizationcited by 5,579
  5. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  6. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  7. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  8. 2021SwinIR: Image Restoration Using Swin Transformercited by 1,869
  9. 2020Rethinking Attention with Performerscited by 1,584
  10. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
  11. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,148
  12. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,096
38 more
  1. 2021Video Swin Transformercited by 1,090
  2. 2018TSM: Temporal Shift Module for Efficient Video Understandingcited by 852
  3. 2018Image Transformercited by 770
  4. 2018Deformable ConvNets v2: More Deformable, Better Resultscited by 666
  5. 2017Video Enhancement with Task-Oriented Flowcited by 612
  6. 2018Free-Form Image Inpainting with Gated Convolutioncited by 594
  7. 2018Generative Image Inpainting with Contextual Attentioncited by 578
  8. 2021CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windowscited by 526
  9. 2019EDVR: Video Restoration with Enhanced Deformable Convolutional Networkscited by 492
  10. 2016Optical Flow Estimation using a Spatial Pyramid Networkcited by 421
  11. 2018Video-to-Video Synthesiscited by 388
  12. 2021BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignmentcited by 340
  13. 2020VirTex: Learning Visual Representations from Textual Annotationscited by 338
  14. 2017Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolationcited by 321
  15. 2020BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyondcited by 303
  16. 2018Learning Blind Video Temporal Consistencycited by 247
  17. 2021Attention Mechanisms in Computer Vision: A Surveycited by 232
  18. 2018YouTube-VOS: Sequence-to-Sequence Video Object Segmentationcited by 228
  19. 2018TDAN: Temporally Deformable Alignment Network for Video Super-Resolutioncited by 225
  20. 2021Keeping Your Eye on the Ball: Trajectory Attention in Video Transformerscited by 224
  21. 2020Consistent Video Depth Estimationcited by 220
  22. 2022Visual Attention Networkcited by 210
  23. 2017Flow-Guided Feature Aggregation for Video Object Detectioncited by 177
  24. 2020Learning Joint Spatial-Temporal Transformations for Video Inpaintingcited by 162
  25. 2019AdaCoF: Adaptive Collaboration of Flows for Video Frame Interpolationcited by 133
  26. 2019Deep Flow-Guided Video Inpaintingcited by 116
  27. 2019Deep Video Inpaintingcited by 107
  28. 2020Flow-edge Guided Video Completioncited by 104
  29. 2021FuseFormer: Fusing Fine-Grained Information in Transformers for Video Inpaintingcited by 94
  30. 2017SegFlow: Joint Learning for Video Object Segmentation and Optical Flowcited by 92
  31. 2020Zooming Slow-Mo: Fast and Accurate One-Stage Space-Time Video Super-Resolutioncited by 91
  32. 2019Free-form Video Inpainting with 3D Gated Convolution and Temporal PatchGANcited by 85
  33. 2020Cascaded Deep Video Deblurring Using Temporal Sharpness Priorcited by 85
  34. 2020Understanding Deformable Alignment in Video Super-Resolutioncited by 78
  35. 2018Video Inpainting by Jointly Learning Temporal Structure and Spatial Detailscited by 57
  36. 2019Copy-and-Paste Networks for Deep Video Inpaintingcited by 57
  37. 2021Temporal Modulation Network for Controllable Space-Time Video Super-Resolutioncited by 54
  38. 2019Onion-Peel Networks for Deep Video Completioncited by 46

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 12; 2023: 26; 2024: 29; 2025: 24; 2026: 20.

111 papers.

Sort:

By area:

  1. 2023Track Anything: Segment Anything Meets Videoscited by 230
  2. 2023ProPainter: Improving Propagation and Transformer for Video Inpaintingcited by 193
  3. 2023AMT: All-Pairs Multi-Field Transforms for Efficient Frame Interpolationcited by 177
  4. 2023Swap Attention in Spatiotemporal Diffusions for Text-to-Video Generationcited by 153
  5. 2025VideoPainter: Any-length Video Inpainting and Editing with Plug-and-Play Context Controlcited by 83
  6. 2022Removing Objects From Neural Radiance Fieldscited by 66
  7. 2023Dense Optical Tracking: Connecting the Dotscited by 58
  8. 2024InFusion: Inpainting 3D Gaussians via Learning Depth Completion from Diffusion Priorcited by 55
  9. 2023Shape-aware Text-driven Layered Video Editingcited by 54
  10. 2025VidBot: Learning Generalizable 3D Actions from In-the-Wild 2D Human Videos for Zero-Shot Robotic Manipulationcited by 46
  11. 2023I'M HOI: Inertia-aware Monocular Capture of 3D Human-Object Interactionscited by 42
  12. 2024Generative Omnimatte: Learning to Decompose Video into Layerscited by 38
38 more
  1. 2024StereoCrafter: Diffusion-based Generation of Long and High-fidelity Stereoscopic 3D from Monocular Videoscited by 36
  2. 2023UMMAFormer: A Universal Multimodal-adaptive Transformer Framework for Temporal Forgery Localizationcited by 33
  3. 2024Towards Language-Driven Video Inpainting via Multimodal Large Language Modelscited by 33
  4. 2024MVInpainter: Learning Multi-View Consistent Inpainting to Bridge 2D and 3D Editingcited by 30
  5. 2024V2A-Mark: Versatile Deep Visual-Audio Watermarking for Manipulation Localization and Copyright Protectioncited by 27
  6. 2023AR2-D2:Training a Robot Without a Robotcited by 26
  7. 2023Mask Propagation for Efficient Video Semantic Segmentationcited by 26
  8. 2023Bidirectionally Deformable Motion Modulation For Video-based Human Pose Transfercited by 25
  9. 2024Ag2Manip: Learning Novel Manipulation Skills with Agent-Agnostic Visual and Action Representationscited by 23
  10. 2022ZITS++: Image Inpainting by Improving the Incremental Transformer on Structural Priorscited by 21
  11. 2024Video Diffusion Models are Strong Video Inpaintercited by 21
  12. 2024SVG: 3D Stereoscopic Video Generation via Denoising Frame Matrixcited by 19
  13. 2026NTIRE 2026 Challenge on Bitstream-Corrupted Video Restoration: Methods and Resultscited by 15
  14. 2023Bitstream-Corrupted Video Recovery: A Novel Benchmark Dataset and Methodcited by 14
  15. 2025Mitty: Diffusion-based Human-to-Robot Video Generationcited by 13
  16. 2023Flow-Guided Diffusion for Video Inpaintingcited by 11
  17. 2023Transformer-based Generative Adversarial Networks in Computer Vision: A Comprehensive Surveycited by 10
  18. 2025TACO: Taming Diffusion for in-the-wild Video Amodal Completioncited by 10
  19. 2025M2SVid: End-to-End Inpainting and Refinement for Monocular-to-Stereo Video Conversioncited by 10
  20. 2025OXE-AugE: A Large-Scale Robot Augmentation of OXE for Scaling Cross-Embodiment Policy Learningcited by 10
  21. 2022VideoFACT: Detecting Video Forgeries Using Attention, Scene Context, and Forensic Tracescited by 9
  22. 2023GRACE: Loss-Resilient Real-Time Video through Neural Codecscited by 9
  23. 2026PISCO: Precise Video Instance Insertion with Sparse Controlcited by 9
  24. 2022Semi-Supervised Video Inpainting with Cycle Consistency Constraintscited by 8
  25. 2022Temporal and Contextual Transformer for Multi-Camera Editing of TV Showscited by 8
  26. 2022FactorMatte: Redefining Video Matting for Re-Composition Taskscited by 8
  27. 2024Semantically Consistent Video Inpainting with Conditional Diffusion Modelscited by 8
  28. 2024ImmersePro: End-to-End Stereo Video Synthesis Via Implicit Disparity Learningcited by 8
  29. 2025H2R-Grounder: A Paired-Data-Free Paradigm for Translating Human Interaction Videos into Physically Grounded Robot Videoscited by 8
  30. 2023UMIFormer: Mining the Correlations between Similar Tokens for Multi-View 3D Reconstructioncited by 7
  31. 2026PAI-Studio: Cinematic Video Background Replacement with Camera-Aware Motioncited by 7
  32. 2023Exploiting Optical Flow Guidance for Transformer-Based Video Inpaintingcited by 6
  33. 2024Transformer-based Image and Video Inpainting: Current Challenges and Future Directionscited by 6
  34. 2024Coherent Video Inpainting Using Optical Flow-Guided Efficient Diffusioncited by 6
  35. 2022VGFlow: Visibility guided Flow Network for Human Reposingcited by 5
  36. 2022Beyond the Field-of-View: Enhancing Scene Visibility and Perception with Clip-Recurrent Transformercited by 5
  37. 2023Deficiency-Aware Masked Transformer for Video Inpaintingcited by 5
  38. 2025Plug-and-Play Versatile Compressed Video Enhancementcited by 5

50 most cited of 111 All 111 (JSON)