# OmniPaint: Mastering Object-Oriented Editing via Disentangled Insertion-Removal Inpainting

arXiv: https://arxiv.org/abs/2503.08677 · HTML: https://mlsanity.com/paper/2503.08677/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models.
- Builds on 53 of its 69 references that are in the map.
- First posted on arXiv 11 Mar 2025.

## Placed in

- Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/)

## Builds on (most cited first, 30 of 53)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [MUSIQ: Multi-scale Image Quality Transformer](https://mlsanity.com/paper/2108.05997/) (2021)
- [COCO-Stuff: Thing and Stuff Classes in Context](https://mlsanity.com/paper/1612.03716/) (2016)
- [Resolution-robust Large Mask Inpainting with Fourier Convolutions](https://mlsanity.com/paper/2109.07161/) (2021)
- [MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessment](https://mlsanity.com/paper/2204.08958/) (2022)
- [Paint by Example: Exemplar-based Image Editing with Diffusion Models](https://mlsanity.com/paper/2211.13227/) (2022)
- [Blended Latent Diffusion](https://mlsanity.com/paper/2206.02779/) (2022)
- [AnyDoor: Zero-shot Object-level Image Customization](https://mlsanity.com/paper/2307.09481/) (2023)
- [Segment Anything in High Quality](https://mlsanity.com/paper/2306.01567/) (2023)
- [OminiControl: Minimal and Universal Control for Diffusion Transformer](https://mlsanity.com/paper/2411.15098/) (2024)
- [Rethinking FID: Towards a Better Evaluation Metric for Image Generation](https://mlsanity.com/paper/2401.09603/) (2023)
- [MAT: Mask-Aware Transformer for Large Hole Image Inpainting](https://mlsanity.com/paper/2203.15270/) (2022)
- [Guiding Instruction-based Image Editing via Multimodal Large Language Models](https://mlsanity.com/paper/2309.17102/) (2023)
- [A Task is Worth One Word: Learning with Task Prompts for High-Quality Versatile Image Inpainting](https://mlsanity.com/paper/2312.03594/) (2023)
- [Deep Image Harmonization](https://mlsanity.com/paper/1703.00069/) (2017)
- [VideoPainter: Any-length Video Inpainting and Editing with Plug-and-Play Context Control](https://mlsanity.com/paper/2503.05639/) (2025)
- [ObjectDrop: Bootstrapping Counterfactuals for Photorealistic Object Removal and Insertion](https://mlsanity.com/paper/2403.18818/) (2024)
- [Inst-Inpaint: Instructing to Remove Objects with Diffusion Models](https://mlsanity.com/paper/2304.03246/) (2023)
- [GP-GAN: Towards Realistic High-Resolution Image Blending](https://mlsanity.com/paper/1703.07195/) (2017)
- [IMPRINT: Generative Object Compositing by Learning Identity-Preserving Representation](https://mlsanity.com/paper/2403.10701/) (2024)

## Built on by (most cited first, 30 of 39)

- [NoHumansRequired: Autonomous High-Quality Image Editing Triplet Mining](https://mlsanity.com/paper/2507.14119/) (2025)
- [Does FLUX Already Know How to Perform Physically Plausible Image Composition?](https://mlsanity.com/paper/2509.21278/) (2025)
- [PhyGDPO: Physics-Aware Groupwise Direct Preference Optimization for Physically Consistent Text-to-Video Generation](https://mlsanity.com/paper/2512.24551/) (2025)
- [MMIG-Bench: Towards Comprehensive and Explainable Evaluation of Multi-Modal Image Generation Models](https://mlsanity.com/paper/2505.19415/) (2025)
- [OmniVCus: Feedforward Subject-driven Video Customization with Multimodal Control Conditions](https://mlsanity.com/paper/2506.23361/) (2025)
- [MIRA: Multimodal Iterative Reasoning Agent for Image Editing](https://mlsanity.com/paper/2511.21087/) (2025)
- [EffectErase: Joint Video Object Removal and Insertion for High-Quality Effect Erasing](https://mlsanity.com/paper/2603.19224/) (2026)
- [Precise Object and Effect Removal with Adaptive Target-Aware Attention](https://mlsanity.com/paper/2505.22636/) (2025)
- [MementoGUI: Learning Agentic Multimodal Memory Control for Long-Horizon GUI Agents](https://mlsanity.com/paper/2605.18652/) (2026)
- [GeoRemover: Removing Objects and Their Causal Visual Artifacts](https://mlsanity.com/paper/2509.18538/) (2025)
- [In-Context Brush: Zero-shot Customized Subject Insertion with Context-Aware Latent Space Manipulation](https://mlsanity.com/paper/2505.20271/) (2025)
- [Credit Where It is Due: Cross-Modality Connectivity Drives Precise Reinforcement Learning for MLLM Reasoning](https://mlsanity.com/paper/2602.11455/) (2026)
- [From Ideal to Real: Stable Video Object Removal under Imperfect Conditions](https://mlsanity.com/paper/2603.09283/) (2026)
- [VideoArgus: Agentic Rubric-Grounded Unified Evaluation for Video Generation and Editing](https://mlsanity.com/paper/2608.05485/) (2026)
- [RePainter: Empowering E-commerce Object Removal via Spatial-matting Reinforcement Learning](https://mlsanity.com/paper/2510.07721/) (2025)
- [RoboDream: Compositional World Models for Scalable Robot Data Synthesis](https://mlsanity.com/paper/2606.02577/) (2026)
- [PixPerfect: Seamless Latent Diffusion Local Editing with Discriminative Pixel-Space Refinement](https://mlsanity.com/paper/2512.03247/) (2025)
- [VIBE: Visual Instruction Based Editor](https://mlsanity.com/paper/2601.02242/) (2026)
- [EasyOmnimatte: Taming Pretrained Inpainting Diffusion Models for End-to-End Video Layered Decomposition](https://mlsanity.com/paper/2512.21865/) (2025)
- [FlashClear: Ultra-Fast Image Content Removal via Efficient Step Distillation and Feature Caching](https://mlsanity.com/paper/2605.09003/) (2026)
- [DORS: Dynamic Attention Routing for Diffusion-based Object Removal in Dense Scenes](https://mlsanity.com/paper/2607.16656/) (2026)
- [BeyondMasks: Evaluating Causal and Physical Consistency in Video Object Removal](https://mlsanity.com/paper/2608.20107/) (2026)
- [EpiCon: Collective Agent Learning through Co-Evolving Multimodal Memory](https://mlsanity.com/paper/2609.37923/) (2026)
- [Mask Consistency Regularization in Object Removal](https://mlsanity.com/paper/2509.10259/) (2025)
- [InsertAnywhere: Geometrically Grounded and Optics-Aware Video Object Insertion](https://mlsanity.com/paper/2512.17504/) (2025)
- [EraseLoRA: MLLM-Driven Foreground Exclusion and Background Subtype Aggregation for Dataset-Free Object Removal](https://mlsanity.com/paper/2512.21545/) (2025)
- [Aligned Stable Inpainting: Mitigating Unwanted Object Insertion and Preserving Color Consistency](https://mlsanity.com/paper/2601.15368/) (2026)
- [Consistency-Preserving Concept Erasure via Unsafe-Safe Pairing and Directional Fisher-weighted Adaptation](https://mlsanity.com/paper/2602.05339/) (2026)
- [PICS: Pairwise Image Compositing with Spatial Interactions](https://mlsanity.com/paper/2603.06873/) (2026)
- [You Only Erase Once: Erasing Anything without Bringing Unexpected Content](https://mlsanity.com/paper/2603.27599/) (2026)

## Abstract (arXiv)

Diffusion-based generative models have revolutionized object-oriented image editing, yet their deployment in realistic object removal and insertion remains hampered by challenges such as the intricate interplay of physical effects and insufficient paired training data. In this work, we introduce OmniPaint, a unified framework that re-conceptualizes object removal and insertion as interdependent processes rather than isolated tasks. Leveraging a pre-trained diffusion prior along with a progressive training pipeline comprising initial paired sample optimization and subsequent large-scale unpaired refinement via CycleFlow, OmniPaint achieves precise foreground elimination and seamless object insertion while faithfully preserving scene geometry and intrinsic properties. Furthermore, our novel CFD metric offers a robust, reference-free evaluation of context consistency and object hallucination, establishing a new benchmark for high-fidelity image editing. Project page: https://yeates.github.io/OmniPaint-Page/