# Monocular Depth Decomposition of Semi-Transparent Volume Renderings

arXiv: https://arxiv.org/abs/2206.13282 · HTML: https://mlsanity.com/paper/2206.13282/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 57 references that are in the map.
- First posted on arXiv 27 Jun 2022.

## Builds on (most cited first, 20 of 20)

- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [Deep Ordinal Regression Network for Monocular Depth Estimation](https://mlsanity.com/paper/1806.02446/) (2018)
- [Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecture](https://mlsanity.com/paper/1411.4734/) (2014)
- [Deeper Depth Prediction with Fully Convolutional Residual Networks](https://mlsanity.com/paper/1606.00373/) (2016)
- [From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimation](https://mlsanity.com/paper/1907.10326/) (2019)
- [Enforcing geometric constraints of virtual normal for depth prediction](https://mlsanity.com/paper/1907.12209/) (2019)
- [The Edge of Depth: Explicit Constraints between Segmentation and Depth](https://mlsanity.com/paper/2004.00171/) (2020)
- [Predicting Sharp and Accurate Occlusion Boundaries in Monocular Depth Estimation Using Displacement Fields](https://mlsanity.com/paper/2002.12730/) (2020)
- [Focus on defocus: bridging the synthetic to real domain gap for depth estimation](https://mlsanity.com/paper/2005.09623/) (2020)
- [Fast Neural Representations for Direct Volume Rendering](https://mlsanity.com/paper/2112.01579/) (2021)
- [DL4SciVis: A State-of-the-Art Survey on Deep Learning for Scientific Visualization](https://mlsanity.com/paper/2204.06504/) (2022)
- [Development and Validation of Deep Learning Algorithms for Detection of Critical Findings in Head CT Scans](https://mlsanity.com/paper/1803.05854/) (2018)
- [Volumetric Isosurface Rendering with Deep Learning-Based Super-Resolution](https://mlsanity.com/paper/1906.06520/) (2019)
- [Domain Decluttering: Simplifying Images to Mitigate Synthetic-Real Domain Shift and Improve Depth Estimation](https://mlsanity.com/paper/2002.12114/) (2020)
- [Differentiable Direct Volume Rendering](https://mlsanity.com/paper/2107.12672/) (2021)
- [Learning Adaptive Sampling and Reconstruction for Volume Visualization](https://mlsanity.com/paper/2007.10093/) (2020)
- [SelfDeco: Self-Supervised Monocular Depth Completion in Challenging Indoor Environments](https://mlsanity.com/paper/2011.04977/) (2020)
- [Deep Volumetric Ambient Occlusion](https://mlsanity.com/paper/2008.08345/) (2020)
- [Deep Direct Volume Rendering: Learning Visual Feature Mappings From Exemplary Images](https://mlsanity.com/paper/2106.05429/) (2021)

## Built on by (most cited first, 1 of 1)

- [Multi-Temporal Frames Projection for Dynamic Processes Fusion in Fluorescence Microscopy](https://mlsanity.com/paper/2601.10392/) (2026)

## Abstract (arXiv)

Neural networks have shown great success in extracting geometric information from color images. Especially, monocular depth estimation networks are increasingly reliable in real-world scenes. In this work we investigate the applicability of such monocular depth estimation networks to semi-transparent volume rendered images. As depth is notoriously difficult to define in a volumetric scene without clearly defined surfaces, we consider different depth computations that have emerged in practice, and compare state-of-the-art monocular depth estimation approaches for these different interpretations during an evaluation considering different degrees of opacity in the renderings. Additionally, we investigate how these networks can be extended to further obtain color and opacity information, in order to create a layered representation of the scene based on a single color image. This layered representation consists of spatially separated semi-transparent intervals that composite to the original input rendering. In our experiments we show that existing approaches to monocular depth estimation can be adapted to perform well on semi-transparent volume renderings, which has several applications in the area of scientific visualization, like re-composition with additional objects and labels or additional shading.