# Interactive Occlusion Boundary Estimation through Exploitation of Synthetic Data

arXiv: https://arxiv.org/abs/2408.15038 · HTML: https://mlsanity.com/paper/2408.15038/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 86 references that are in the map.
- First posted on arXiv 27 Aug 2024.

## Builds on (most cited first, 30 of 40)

- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [3D-FUTURE: 3D Furniture shape with TextURE](https://mlsanity.com/paper/2009.09633/) (2020)
- [DIODE: A Dense Indoor and Outdoor DEpth Dataset](https://mlsanity.com/paper/1908.00463/) (2019)
- [Reviving Iterative Training with Mask Guidance for Interactive Segmentation](https://mlsanity.com/paper/2102.06583/) (2021)
- [Deep Interactive Object Selection](https://mlsanity.com/paper/1603.04042/) (2016)
- [Evaluation of CNN-based Single-Image Depth Estimation Methods](https://mlsanity.com/paper/1805.01328/) (2018)
- [Deep Extreme Cut: From Extreme Points to Object Segmentation](https://mlsanity.com/paper/1711.09081/) (2017)
- [Interactive Medical Image Segmentation using Deep Learning with Image-specific Fine-tuning](https://mlsanity.com/paper/1710.04043/) (2017)
- [SimpleClick: Interactive Image Segmentation with Simple Vision Transformers](https://mlsanity.com/paper/2210.11006/) (2022)
- [Semantic Amodal Segmentation](https://mlsanity.com/paper/1509.01329/) (2015)
- [FocalClick: Towards Practical Interactive Image Segmentation](https://mlsanity.com/paper/2204.02574/) (2022)
- [High-Quality Entity Segmentation](https://mlsanity.com/paper/2211.05776/) (2022)
- [InteriorNet: Mega-scale Multi-sensor Photo-realistic Indoor Scenes Dataset](https://mlsanity.com/paper/1809.00716/) (2018)
- [SharpNet: Fast and Accurate Recovery of Occluding Contours in Monocular Depth Estimation](https://mlsanity.com/paper/1905.08598/) (2019)
- [Occluded Human Mesh Recovery](https://mlsanity.com/paper/2203.13349/) (2022)
- [MIDeepSeg: Minimally Interactive Segmentation of Unseen Objects from Medical Images Using Deep Learning](https://mlsanity.com/paper/2104.12166/) (2021)
- [Predicting Sharp and Accurate Occlusion Boundaries in Monocular Depth Estimation Using Displacement Fields](https://mlsanity.com/paper/2002.12730/) (2020)
- [EdgeFlow: Achieving Practical Interactive Segmentation with Edge-Guided Flow](https://mlsanity.com/paper/2109.09406/) (2021)
- [Interactive Full Image Segmentation by Considering All Regions Jointly](https://mlsanity.com/paper/1812.01888/) (2018)
- [RINDNet: Edge Detection for Discontinuity in Reflectance, Illumination, Normal and Depth](https://mlsanity.com/paper/2108.00616/) (2021)
- [InterFormer: Real-time Interactive Image Segmentation](https://mlsanity.com/paper/2304.02942/) (2023)
- [Occlusion-shared and Feature-separated Network for Occlusion Relationship Reasoning](https://mlsanity.com/paper/1908.05898/) (2019)
- [A Fully Convolutional Two-Stream Fusion Network for Interactive Image Segmentation](https://mlsanity.com/paper/1807.02480/) (2018)
- [Interactive Multi-Class Tiny-Object Detection](https://mlsanity.com/paper/2203.15266/) (2022)
- [GraCo: Granularity-Controllable Interactive Segmentation](https://mlsanity.com/paper/2405.00587/) (2024)
- [AdaptiveClick: Clicks-aware Transformer with Adaptive Focal Loss for Interactive Image Segmentation](https://mlsanity.com/paper/2305.04276/) (2023)
- [ScribbleSeg: Scribble-based Interactive Image Segmentation](https://mlsanity.com/paper/2303.11320/) (2023)
- [NeTO:Neural Reconstruction of Transparent Objects with Self-Occlusion Aware Refraction-Tracing](https://mlsanity.com/paper/2303.11219/) (2023)
- [Pixel-Pair Occlusion Relationship Map(P2ORM): Formulation, Inference & Application](https://mlsanity.com/paper/2007.12088/) (2020)

## Built on by (most cited first, 2 of 2)

- [Occlusion Boundary and Depth: Mutual Enhancement via Multi-Task Learning](https://mlsanity.com/paper/2505.21231/) (2025)
- [RealOOB: A Definition-Consistent Real-World Oriented Occlusion Boundary Benchmark](https://mlsanity.com/paper/2608.30820/) (2026)

## Abstract (arXiv)

Occlusion boundaries (OBs) geometrically localize occlusion events in 2D images and provide critical cues for scene understanding. In this paper, we present the first systematic study of Interactive Occlusion Boundary Estimation (IOBE), introducing MStextsuperscript3PE, a novel multi-scribble-guided deep-learning framework that advances IOBE through two key innovations: (1) an intuitive multi-scribble interaction mechanism, and (2) a 3-encoding-path network enhanced with multi-scale strip convolutions. Our MStextsuperscript3PE surpasses adapted baselines from seven state-of-the-art interactive segmentation methods, and demonstrates strong potential for OB benchmark construction through our real-user experiment. Besides, to address the scarcity of well-annotated real-world data, we propose using synthetic data for training IOBE models, and developed Mesh2OB, the first automated tool for generating precise ground-truth OBs from 3D scenes with self-occlusions explicitly handled, enabling creation of the OB-FUTURE synthetic benchmark that facilitates generalizable training without domain adaptation. Finally, we introduce OB-LIGM, a high-quality real-world benchmark comprising 120 meticulously annotated high-resolution images advancing evaluation standards in OB research. Source code and resources are available at https://github.com/xul-ops/IOBE.