mlsanityData to 7 Oct 2026

MAT: Mask-Aware Transformer for Large Hole Image Inpainting

arXiv 2203.15270 ↗First posted 29 Mar 2022cs.CVcited by 216

Not on the map. Why?

Cited by 54 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recent studies have shown the importance of modeling long-range interactions in the inpainting problem. To achieve this goal, existing approaches exploit either standalone attention techniques or transformers, but usually under a low resolution in consideration of computational cost. In this paper, we present a novel transformer-based model for large hole inpainting, which unifies the merits of transformers and convolutions to efficiently process high-resolution images. We carefully design each component of our framework to guarantee the high fidelity and diversity of recovered images. Specifically, we customize an inpainting-oriented transformer block, where the attention module aggregates non-local information only from partial valid tokens, indicated by a dynamic mask. Extensive experiments demonstrate the state-of-the-art performance of the new model on multiple benchmark datasets. Code is released at https://github.com/fenglinglwb/MAT.

Key facts. Cited by 216 papers in the map (data through 7 Oct 2026). Builds on 54 of its 80 references that are in the map. First posted on arXiv 29 Mar 2022.

Builds on

54 of 80 references.

References · 54

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  6. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  7. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  8. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  9. 2022Generative Adversarial Networkscited by 5,603
  10. 2016Layer Normalizationcited by 5,579
  11. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  12. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
38 more
  1. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
  2. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
  3. 2017Non-local Neural Networkscited by 2,290
  4. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,898
  5. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
  6. 2021CvT: Introducing Convolutions to Vision Transformerscited by 946
  7. 2021Resolution-robust Large Mask Inpainting with Fourier Convolutionscited by 730
  8. 2018Which Training Methods for GANs do actually Converge?cited by 672
  9. 2021Do Vision Transformers See Like Convolutional Neural Networks?cited by 671
  10. 2018Free-Form Image Inpainting with Gated Convolutioncited by 594
  11. 2018Generative Image Inpainting with Contextual Attentioncited by 578
  12. 2018Image Inpainting for Irregular Holes Using Partial Convolutionscited by 505
  13. 2021Early Convolutions Help Transformers See Bettercited by 395
  14. 2017Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradientscited by 236
  15. 2021Large Scale Image Completion via Co-Modulated Generative Adversarial Networkscited by 219
  16. 2016EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesiscited by 177
  17. 2019Pluralistic Image Completioncited by 162
  18. 2019EdgeConnect: Generative Image Inpainting with Adversarial Edge Learningcited by 161
  19. 2021High-Fidelity Pluralistic Image Completion with Transformerscited by 134
  20. 2020Contextual Residual Aggregation for Ultra High-Resolution Image Inpaintingcited by 124
  21. 2019StructureFlow: Image Inpainting via Structure-aware Appearance Flowcited by 118
  22. 2020Recurrent Feature Reasoning for Image Inpaintingcited by 107
  23. 2019SC-FEGAN: Face Editing Generative Adversarial Network with User's Sketch and Colorcited by 105
  24. 2020Rethinking Image Inpainting via a Mutual Encoder-Decoder with Feature Equalizationscited by 102
  25. 2019Coherent Semantic Attention for Image Inpaintingcited by 97
  26. 2020Bringing Old Photos Back to Lifecited by 95
  27. 2019Learning Pyramid-Context Encoder Network for High-Quality Image Inpaintingcited by 94
  28. 2019Foreground-aware Image Inpaintingcited by 90
  29. 2021Aggregated Contextual Transformations for High-Resolution Image Inpaintingcited by 89
  30. 2018Shift-Net: Image Inpainting via Deep Feature Rearrangementcited by 87
  31. 2018Image Inpainting via Generative Multi-column Convolutional Neural Networkscited by 81
  32. 2021Diverse Image Inpainting with Bidirectional and Autoregressive Transformerscited by 80
  33. 2020High-Resolution Image Inpainting with Iterative Confidence Feedback and Guided Upsamplingcited by 79
  34. 2019Image Inpainting with Learnable Bidirectional Attention Mapscited by 64
  35. 2018SPG-Net: Segmentation Prediction and Guidance Network for Image Inpaintingcited by 58
  36. 2019Onion-Peel Networks for Deep Video Completioncited by 46
  37. 2021Image Inpainting by End-to-End Cascaded Refinement with Mask Awarenesscited by 35
  38. 2018On Self Modulation for Generative Adversarial Networkscited by 33

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 22; 2023: 67; 2024: 60; 2025: 48; 2026: 19.

216 papers.

Sort:

By area:

  1. 2023DiffIR: Efficient Diffusion Model for Image Restorationcited by 253
  2. 2023Inpaint Anything: Segment Anything Meets Image Inpaintingcited by 233
  3. 2023SmartEdit: Exploring Complex Instruction-based Image Editing with Multimodal Large Language Modelscited by 174
  4. 2023Flow Matching in Latent Spacecited by 155
  5. 2022SPIn-NeRF: Multiview Segmentation and Perceptual Inpainting with Neural Radiance Fieldscited by 144
  6. 2023Diffusion Models for Image Restoration and Enhancement: A Comprehensive Surveycited by 108
  7. 2023A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?cited by 84
  8. 2023Inst-Inpaint: Instructing to Remove Objects with Diffusion Modelscited by 66
  9. 2023A Comprehensive Survey on Segment Anything Model for Vision and Beyondcited by 64
  10. 2024Zero-shot Image Editing with Reference Imitationcited by 64
  11. 2023Multimodal Garment Designer: Human-Centric Latent Diffusion Models for Fashion Image Editingcited by 62
  12. 2023The Blessing of Randomness: SDE Beats ODE in General Diffusion-based Image Editingcited by 61
38 more
  1. 2024InFusion: Inpainting 3D Gaussians via Learning Depth Completion from Diffusion Priorcited by 55
  2. 2023Persistent Nature: A Generative Model of Unbounded 3D Worldscited by 44
  3. 2024Deep Learning-based Image and Video Inpainting: A Surveycited by 44
  4. 2023Perceptual Artifacts Localization for Image Synthesis Taskscited by 43
  5. 2023Reference-guided Controllable Inpainting of Neural Radiance Fieldscited by 40
  6. 2024Structure Matters: Tackling the Semantic Discrepancy in Diffusion Models for Image Inpaintingcited by 40
  7. 2025OmniPaint: Mastering Object-Oriented Editing via Disentangled Insertion-Removal Inpaintingcited by 38
  8. 2023ArtiFact: A Large-Scale Dataset with Artificial and Factual Images for Generalizable and Robust Synthetic Image Detectioncited by 35
  9. 2024FreeEdit: Mask-free Reference-based Image Editing with Multi-modal Instructioncited by 35
  10. 2025OmniEraser: Remove Objects and Their Effects in Images with Paired Video-Frame Datacited by 35
  11. 2023CoordFill: Efficient High-Resolution Image Inpainting via Parameterized Coordinate Queryingcited by 34
  12. 2022Masked Lip-Sync Prediction by Audio-Visual Contextual Exploitation in Transformerscited by 33
  13. 2024Towards Language-Driven Video Inpainting via Multimodal Large Language Modelscited by 33
  14. 2024MVInpainter: Learning Multi-View Consistent Inpainting to Bridge 2D and 3D Editingcited by 30
  15. 2025ObjectMover: Generative Object Movement with Video Priorcited by 30
  16. 2023UniGS: Unified Representation for Image Generation and Segmentationcited by 29
  17. 2023LeftRefill: Filling Right Canvas based on Left Reference through Generalized Text-to-Image Diffusion Modelcited by 25
  18. 2023DiffI2I: Efficient Diffusion Model for Image-to-Image Translationcited by 24
  19. 2024CharacterFactory: Sampling Consistent Characters with GANs for Diffusion Modelscited by 24
  20. 2025SmartEraser: Remove Anything from Images using Masked-Region Guidancecited by 24
  21. 2023Behind the Magic, MERLIM: Multi-modal Evaluation Benchmark for Large Image-Language Modelscited by 23
  22. 2025RORem: Training a Robust Object Remover with Human-in-the-Loopcited by 23
  23. 2023Generated Distributions Are All You Need for Membership Inference Attacks Against Generative Modelscited by 22
  24. 2023Exploring the Naturalness of AI-Generated Imagescited by 22
  25. 2022ZITS++: Image Inpainting by Improving the Incremental Transformer on Structural Priorscited by 21
  26. 2023Rethinking Image Editing Detection in the Era of Generative AI Revolutioncited by 21
  27. 2024RefFusion: Reference Adapted Diffusion Models for 3D Scene Inpaintingcited by 21
  28. 2023Towards Enhanced Image Inpainting: Mitigating Unwanted Object Insertion and Preserving Color Consistencycited by 19
  29. 2024CLIPAway: Harmonizing Focused Embeddings for Removing Objects via Diffusion Modelscited by 19
  30. 2024RAD: Region-Aware Diffusion Models for Image Inpaintingcited by 19
  31. 2022Deep Face Restoration: A Surveycited by 17
  32. 2023Deep Equilibrium Diffusion Restoration with Parallel Samplingcited by 17
  33. 2024PrefPaint: Aligning Image Inpainting Diffusion Model with Human Preferencecited by 17
  34. 2022Learning Prior Feature and Attention Enhanced Image Inpaintingcited by 16
  35. 2022Understanding Embodied Reference with Touch-Line Transformercited by 16
  36. 2023Uncertainty Quantification via Neural Posterior Principal Componentscited by 16
  37. 2023Magicremover: Tuning-free Text-guided Image inpainting with Diffusion Modelscited by 16
  38. 2025Zooming In on Fakes: A Novel Dataset for Localized AI-Generated Image Detection with Forgery Amplification Approachcited by 16

50 most cited of 216 All 216 (JSON)