mlsanityData to 7 Oct 2026

Inpaint Anything: Segment Anything Meets Image Inpainting

arXiv 2304.06790 ↗First posted 13 Apr 2023cs.CVcited by 233

Not on the map. Why?

Cited by 54 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Modern image inpainting systems, despite the significant progress, often struggle with mask selection and holes filling. Based on Segment-Anything Model (SAM), we make the first attempt to the mask-free image inpainting and propose a new paradigm of ``clicking and filling'', which is named as Inpaint Anything (IA). The core idea behind IA is to combine the strengths of different models in order to build a very powerful and user-friendly pipeline for solving inpainting-related problems. IA supports three main features: (i) Remove Anything: users could click on an object and IA will remove it and smooth the ``hole'' with the context; (ii) Fill Anything: after certain objects removal, users could provide text-based prompts to IA, and then it will fill the hole with the corresponding generative content via driving AIGC models like Stable Diffusion; (iii) Replace Anything: with IA, users have another option to retain the click-selected object and replace the remaining background with the newly generated scenes. We are also very willing to help everyone share and promote new projects based on our Inpaint Anything (IA). Our codes are available at https://github.com/geekyutao/Inpaint-Anything.

Key facts. Cited by 233 papers in the map (data through 7 Oct 2026). Builds on 10 of its 13 references that are in the map. First posted on arXiv 13 Apr 2023.

Builds on

10 of 13 references.

References · 10

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2023Segment Anythingcited by 8,158
  4. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  5. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
  6. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,419
  7. 2021Resolution-robust Large Mask Inpainting with Fourier Convolutionscited by 730
  8. 2022MAT: Mask-Aware Transformer for Large Hole Image Inpaintingcited by 216
  9. 2019Image inpainting: A reviewcited by 89
  10. 2022Incremental Transformer Structure Enhanced Image Inpainting with Masking Positional Encodingcited by 83

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 54; 2024: 93; 2025: 66; 2026: 20.

233 papers.

Sort:

By area:

  1. 2023AnyDoor: Zero-shot Object-level Image Customizationcited by 411
  2. 2023LangSplat: 3D Language Gaussian Splattingcited by 410
  3. 2023Faster Segment Anything: Towards Lightweight SAM for Mobile Applicationscited by 345
  4. 2023SPHINX: The Joint Mixing of Weights, Tasks, and Visual Embeddings for Multi-modal Large Language Modelscited by 283
  5. 2023Multimodal Foundation Models: From Specialists to General-Purpose Assistantscited by 247
  6. 2024Diffusion Model-Based Image Editing: A Surveycited by 212
  7. 2024BrushNet: A Plug-and-Play Image Inpainting Model with Decomposed Dual-Branch Diffusioncited by 200
  8. 2023EfficientSAM: Leveraged Masked Image Pretraining for Efficient Segment Anythingcited by 187
  9. 2024A Survey on Diffusion Models for Inverse Problemscited by 173
  10. 2024PhysGen: Rigid-Body Physics-Grounded Image-to-Video Generationcited by 169
  11. 2023SAM-6D: Segment Anything Model Meets Zero-Shot 6D Object Pose Estimationcited by 115
  12. 2023Diffusion Models for Image Restoration and Enhancement: A Comprehensive Surveycited by 108
38 more
  1. 2023SAM on Medical Images: A Comprehensive Study on Three Prompt Modescited by 81
  2. 2023Review of Large Vision Models and Visual Prompt Engineeringcited by 77
  3. 2023Dynamic Prompt Learning: Addressing Cross-Attention Leakage for Text-Based Image Editingcited by 76
  4. 2024VRP-SAM: SAM with Visual Reference Promptcited by 75
  5. 2023Foundational Models in Medical Imaging: A Comprehensive Survey and Future Visioncited by 74
  6. 2023A Comprehensive Survey on Segment Anything Model for Vision and Beyondcited by 64
  7. 2023InstructEdit: Improving Automatic Masks for Diffusion-based Image Editing With User Instructionscited by 64
  8. 2024Zero-shot Image Editing with Reference Imitationcited by 64
  9. 2024EVF-SAM: Early Vision-Language Fusion for Text-Prompted Segment Anything Modelcited by 63
  10. 2023MagicStick: Controllable Video Editing via Control Handle Transformationscited by 62
  11. 2023Segment Anything in 3D with Radiance Fieldscited by 61
  12. 2024InFusion: Inpainting 3D Gaussians via Learning Depth Completion from Diffusion Priorcited by 55
  13. 2023PanoDiffusion: 360-degree Panorama Outpainting via Diffusioncited by 50
  14. 2023A Survey on Segment Anything Model (SAM): Vision Foundation Model Meets Prompt Engineeringcited by 45
  15. 2024TinyFusion: Diffusion Transformers Learned Shallowcited by 44
  16. 2025Reflect-DiT: Inference-Time Scaling for Text-to-Image Diffusion Transformers via In-Context Reflectioncited by 42
  17. 2023MobileSAMv2: Faster Segment Anything to Everythingcited by 41
  18. 2023Matte Anything: Interactive Natural Image Matting with Segment Anything Modelscited by 39
  19. 2024Paint by Inpaint: Learning to Add Image Objects by Removing Them Firstcited by 39
  20. 2024Efficient Diffusion Models: A Comprehensive Survey from Principles to Practicescited by 39
  21. 2025VEGGIE: Instructional Editing and Reasoning Video Concepts with Grounded Generationcited by 39
  22. 2024RobustSAM: Segment Anything Robustly on Degraded Imagescited by 36
  23. 2023TinySAM: Pushing the Envelope for Efficient Segment Anything Modelcited by 33
  24. 2024Towards Language-Driven Video Inpainting via Multimodal Large Language Modelscited by 33
  25. 2024Diving into Underwater: Segment Anything Model Guided Underwater Salient Instance Segmentation and A Large-scale Datasetcited by 33
  26. 2024Run-time Observation Interventions Make Vision-Language-Action Models More Visually Robustcited by 33
  27. 2024PTQ4SAM: Post-Training Quantization for Segment Anythingcited by 32
  28. 2024BrushEdit: All-In-One Image Inpainting and Editingcited by 31
  29. 2023Robustness of SAM: Segment Anything Under Corruptions and Beyondcited by 30
  30. 2023Unifying Image Processing as Visual Prompting Question Answeringcited by 29
  31. 2024UV-SAM: Adapting Segment Anything Model for Urban Village Identificationcited by 29
  32. 2024SAGD: Boundary-Enhanced Segment Anything in 3D Gaussian via Gaussian Decompositioncited by 29
  33. 2025Qwen-Image-Layered: Towards Inherent Editability via Layer Decompositioncited by 29
  34. 2024Decoupling Pixel Flipping and Occlusion Strategy for Consistent XAI Benchmarkscited by 28
  35. 2023Differential Diffusion: Giving Each Pixel Its Strengthcited by 27
  36. 2024GoodSAM: Bridging Domain and Capacity Gaps via Segment Anything Model for Distortion-aware Panoramic Semantic Segmentationcited by 27
  37. 2025ArtiScene: Language-Driven Artistic 3D Scene Generation Through Image Intermediarycited by 27
  38. 2023On the Robustness of Segment Anythingcited by 24

50 most cited of 233 All 233 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Insert Anything: Image Insertion via In-Context Editing in DiT
  2. 2024Fill in the ____ (a Diffusion-based Image Inpainting Pipeline)
  3. 2025OmniPaint: Mastering Object-Oriented Editing via Disentangled Insertion-Removal Inpainting
  4. 2022SmartBrush: Text and Shape Guided Object Inpainting with Diffusion Model
  5. 2024Diffree: Text-Guided Shape Free Object Inpainting with Diffusion Model
  6. 2023Inst-Inpaint: Instructing to Remove Objects with Diffusion Models
  7. 2024InVi: Object Insertion In Videos Using Off-the-Shelf Diffusion Models
  8. 2023Inpaint3D: 3D Scene Content Generation using 2D Inpainting Diffusion