# Mixed Reality Depth Contour Occlusion Using Binocular Similarity Matching and Three-dimensional Contour Optimisation

arXiv: https://arxiv.org/abs/2203.02300 · HTML: https://mlsanity.com/paper/2203.02300/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 4 of its 36 references that are in the map.
- First posted on arXiv 4 Mar 2022.

## Builds on (most cited first, 4 of 4)

- [Consistent Video Depth Estimation](https://mlsanity.com/paper/2004.15021/) (2020)
- [Fast Optical Flow using Dense Inverse Search](https://mlsanity.com/paper/1603.03590/) (2016)
- [Occlusion-shared and Feature-separated Network for Occlusion Relationship Reasoning](https://mlsanity.com/paper/1908.05898/) (2019)
- [GrabAR: Occlusion-aware Grabbing Virtual Objects in AR](https://mlsanity.com/paper/1912.10637/) (2019)

## Abstract (arXiv)

Mixed reality applications often require virtual objects that are partly occluded by real objects. However, previous research and commercial products have limitations in terms of performance and efficiency. To address these challenges, we propose a novel depth contour occlusion (DCO) algorithm. The proposed method is based on the sensitivity of contour occlusion and a binocular stereoscopic vision device. In this method, a depth contour map is combined with a sparse depth map obtained from a two-stage adaptive filter area stereo matching algorithm and the depth contour information of the objects extracted by a digital image stabilisation optical flow method. We also propose a quadratic optimisation model with three constraints to generate an accurate dense map of the depth contour for high-quality real-virtual occlusion. The whole process is accelerated by GPU. To evaluate the effectiveness of the algorithm, we demonstrate a time con-sumption statistical analysis for each stage of the DCO algorithm execution. To verify the relia-bility of the real-virtual occlusion effect, we conduct an experimental analysis on single-sided, enclosed, and complex occlusions; subsequently, we compare it with the occlusion method without quadratic optimisation. With our GPU implementation for real-time DCO, the evaluation indicates that applying the presented DCO algorithm can enhance the real-time performance and the visual quality of real-virtual occlusion.