Open-World Amodal Appearance Completion
Not on the map. Why?
Abstract (arXiv)
Understanding and reconstructing occluded objects is a challenging problem, especially in open-world scenarios where categories and contexts are diverse and unpredictable. Traditional methods, however, are typically restricted to closed sets of object categories, limiting their use in complex, open-world scenes. We introduce Open-World Amodal Appearance Completion, a training-free framework that expands amodal completion capabilities by accepting flexible text queries as input. Our approach generalizes to arbitrary objects specified by both direct terms and abstract queries. We term this capability reasoning amodal completion, where the system reconstructs the full appearance of the queried object based on the provided image and language query. Our framework unifies segmentation, occlusion analysis, and inpainting to handle complex occlusions and generates completed objects as RGBA elements, enabling seamless integration into applications such as 3D reconstruction and image editing. Extensive evaluations demonstrate the effectiveness of our approach in generalizing to novel objects and occlusions, establishing a new benchmark for amodal completion in open-world settings. The code and datasets will be released after paper acceptance.
Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 34 of its 45 references that are in the map. First posted on arXiv 20 Nov 2024.
Builds on
34 of 45 references.
References · 34
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
- 2023Segment Anythingcited by 8,158
- 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
- 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
- 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
- 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
- 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 914
- 2023LISA: Reasoning Segmentation via Large Language Modelcited by 770
- 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 704
- 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Modelcited by 516
22 more
- 2023VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Taskscited by 499
- 2022Text2LIVE: Text-Driven Layered Image and Video Editingcited by 312
- 2024Transparent Image Layer Diffusion using Latent Transparencycited by 111
- 2015Semantic Amodal Segmentationcited by 102
- 2020Self-Supervised Scene De-occlusioncited by 96
- 2024pix2gestalt: Amodal Segmentation by Synthesizing Wholescited by 70
- 2024Language-Grounded Dynamic Scene Graphs for Interactive Object Search with Mobile Manipulationcited by 69
- 2023Open-Set Image Tagging with Multi-Grained Text Supervisioncited by 67
- 2018Learning to See the Invisible: End-to-End Trainable Amodal Instance Segmentationcited by 62
- 2017SeGAN: Segmenting and Generating the Invisiblecited by 60
- 2023Amodal Ground Truth and Completion in the Wildcited by 43
- 2023Amodal Completion via Progressive Mixed Context Diffusioncited by 38
- 2024MULAN: A Multi Layer Annotated Dataset for Controllable Text-to-Image Generationcited by 38
- 2021Human De-occlusion: Invisible Perception and Recovery for Humanscited by 31
- 2021Instance-wise Occlusion and Depth Orders in Natural Scenescited by 31
- 2019Object-Driven Multi-Layer Scene Decomposition From a Single Imagecited by 26
- 2019Visualizing the Invisible: Occluded Vehicle Segmentation and Recoverycited by 24
- 2021Visiting the Invisible: Layer-by-Layer Completed Scene Decompositioncited by 23
- 2019How to make a pizza: Learning a compositional layer-based GAN modelcited by 21
- 2022Image Amodal Completion: A Surveycited by 18
- 2024Object-level Scene Deocclusioncited by 17
- 2021PanoDR: Spherical Panorama Diminished Reality for Indoor Scenescited by 13
Built on by
Citing papers per year: 2025: 8; 2026: 10.
18 papers.
Sort:
By area:
- 2025Amodal3R: Amodal 3D Reconstruction from Occluded 2D Imagescited by 52
- 2025CAPTURe: Evaluating Spatial Reasoning in Vision Language Models via Occluded Object Countingcited by 36
- 2025PSDiffusion: Harmonized Multi-Layer Image Generation via Layout and Appearance Alignmentcited by 11
- 2026PISCO: Precise Video Instance Insertion with Sparse Controlcited by 9
- 2025GENA3D: Generative Amodal 3D Modeling by Bridging 2D Priors and 3D Coherencecited by 4
- 2025Unveiling the Invisible: Reasoning Complex Occlusions Amodally with AURAcited by 3
- 2026AmodalSVG: Amodal Image Vectorization via Semantic Layer Peelingcited by 3
- 2025GPT-4 for Occlusion Order Recoverycited by 1
- 2025Reasoning-Driven Amodal Completion: Collaborative Agents and Perceptual Evaluationcited by 1
- 2026RelaxFlow: Text-Driven Amodal 3D Generationcited by 1
- 2026Learnable Shape Prototypes with Occlusion-Geometry-Guided Injection for Amodal Instance Segmentationcited by 1
- 2025Extreme Amodal Face Detectioncited by 0
6 more
- 2026PICS: Pairwise Image Compositing with Spatial Interactionscited by 0
- 2026Integrating Multimodal Large Language Model Knowledge into Amodal Completioncited by 0
- 2026LimeCross: Context-Conditioned Layered Image Editing with Structural Consistencycited by 0
- 2026SceneForge: Structured World Supervision from 3D Interventionscited by 0
- 2026Foreseeing the Invisible: Amodal Reconstruction of Leaf Fossil Imagescited by 0
- 2026Absence is Presence: Understanding Visual Scene Negative Events Under Safety Cognitive Constraintcited by 0
- 2026Absence is Presence: Understanding Visual Scene Negative Events Under Safety Cognitive Constraint17 Sep 2026
- 2026Foreseeing the Invisible: Amodal Reconstruction of Leaf Fossil Images5 Aug 2026
- 2026Learnable Shape Prototypes with Occlusion-Geometry-Guided Injection for Amodal Instance Segmentation23 May 2026
- 2026SceneForge: Structured World Supervision from 3D Interventions14 May 2026
- 2026LimeCross: Context-Conditioned Layered Image Editing with Structural Consistency11 May 2026
- 2026AmodalSVG: Amodal Image Vectorization via Semantic Layer Peeling13 Apr 2026
- 2026Integrating Multimodal Large Language Model Knowledge into Amodal Completion30 Mar 2026
- 2026PICS: Pairwise Image Compositing with Spatial Interactions6 Mar 2026
- 2026RelaxFlow: Text-Driven Amodal 3D Generation5 Mar 2026
- 2026PISCO: Precise Video Instance Insertion with Sparse Control9 Feb 2026
- 2025Reasoning-Driven Amodal Completion: Collaborative Agents and Perceptual Evaluation24 Dec 2025
- 2025GENA3D: Generative Amodal 3D Modeling by Bridging 2D Priors and 3D Coherence26 Nov 2025
6 more
- 2025Extreme Amodal Face Detection8 Oct 2025
- 2025GPT-4 for Occlusion Order Recovery26 Sep 2025
- 2025PSDiffusion: Harmonized Multi-Layer Image Generation via Layout and Appearance Alignment16 May 2025
- 2025CAPTURe: Evaluating Spatial Reasoning in Vision Language Models via Occluded Object Counting21 Apr 2025
- 2025Amodal3R: Amodal 3D Reconstruction from Occluded 2D Images17 Mar 2025
- 2025Unveiling the Invisible: Reasoning Complex Occlusions Amodally with AURA13 Mar 2025
- 2025Amodal3R: Amodal 3D Reconstruction from Occluded 2D Imagescited by 52
- 2025PSDiffusion: Harmonized Multi-Layer Image Generation via Layout and Appearance Alignmentcited by 11
- 2025GENA3D: Generative Amodal 3D Modeling by Bridging 2D Priors and 3D Coherencecited by 4
- 2026AmodalSVG: Amodal Image Vectorization via Semantic Layer Peelingcited by 3
- 2025GPT-4 for Occlusion Order Recoverycited by 1
- 2026RelaxFlow: Text-Driven Amodal 3D Generationcited by 1
- 2026Learnable Shape Prototypes with Occlusion-Geometry-Guided Injection for Amodal Instance Segmentationcited by 1
- 2025Extreme Amodal Face Detectioncited by 0
- 2026LimeCross: Context-Conditioned Layered Image Editing with Structural Consistencycited by 0
- 2026SceneForge: Structured World Supervision from 3D Interventionscited by 0
- 2026Foreseeing the Invisible: Amodal Reconstruction of Leaf Fossil Imagescited by 0
- 2026Absence is Presence: Understanding Visual Scene Negative Events Under Safety Cognitive Constraintcited by 0
- 2026Absence is Presence: Understanding Visual Scene Negative Events Under Safety Cognitive Constraint17 Sep 2026
- 2026Foreseeing the Invisible: Amodal Reconstruction of Leaf Fossil Images5 Aug 2026
- 2026Learnable Shape Prototypes with Occlusion-Geometry-Guided Injection for Amodal Instance Segmentation23 May 2026
- 2026SceneForge: Structured World Supervision from 3D Interventions14 May 2026
- 2026LimeCross: Context-Conditioned Layered Image Editing with Structural Consistency11 May 2026
- 2026AmodalSVG: Amodal Image Vectorization via Semantic Layer Peeling13 Apr 2026
- 2026RelaxFlow: Text-Driven Amodal 3D Generation5 Mar 2026
- 2025GENA3D: Generative Amodal 3D Modeling by Bridging 2D Priors and 3D Coherence26 Nov 2025
- 2025Extreme Amodal Face Detection8 Oct 2025
- 2025GPT-4 for Occlusion Order Recovery26 Sep 2025
- 2025PSDiffusion: Harmonized Multi-Layer Image Generation via Layout and Appearance Alignment16 May 2025
- 2025Amodal3R: Amodal 3D Reconstruction from Occluded 2D Images17 Mar 2025