# Deep Supervision with Shape Concepts for Occlusion-Aware 3D Object Parsing

arXiv: https://arxiv.org/abs/1612.02699 · HTML: https://mlsanity.com/paper/1612.02699/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Dec 2016.

## Built on by (most cited first, 19 of 19)

- [Unsupervised Action Segmentation by Joint Representation Learning and Online Clustering](https://mlsanity.com/paper/2105.13353/) (2021)
- [Voxel-based 3D Detection and Reconstruction of Multiple Objects from a Single Image](https://mlsanity.com/paper/2111.03098/) (2021)
- [Contrastive Deep Supervision](https://mlsanity.com/paper/2207.05306/) (2022)
- [Timestamp-Supervised Action Segmentation with Graph Convolutional Networks](https://mlsanity.com/paper/2206.15031/) (2022)
- [Permutation-Aware Action Segmentation via Unsupervised Frame-to-Segment Alignment](https://mlsanity.com/paper/2305.19478/) (2023)
- [HoW-3D: Holistic 3D Wireframe Perception from a Single Image](https://mlsanity.com/paper/2208.06999/) (2022)
- [Joint Self-Supervised Video Alignment and Action Segmentation](https://mlsanity.com/paper/2503.16832/) (2025)
- [A Comprehensive Review on Deep Supervision: Theories and Applications](https://mlsanity.com/paper/2207.02376/) (2022)
- [PQA: Perceptual Question Answering](https://mlsanity.com/paper/2104.03589/) (2021)
- [Occlusion-Robust Object Pose Estimation with Holistic Representation](https://mlsanity.com/paper/2110.11636/) (2021)
- [Action Segmentation Using 2D Skeleton Heatmaps and Multi-Modality Fusion](https://mlsanity.com/paper/2309.06462/) (2023)
- [Weakly But Deeply Supervised Occlusion-Reasoned Parametric Road Layouts](https://mlsanity.com/paper/2104.06730/) (2021)
- [Single Image 3D Object Estimation with Primitive Graph Networks](https://mlsanity.com/paper/2109.04153/) (2021)
- [WALT3D: Generating Realistic Training Data from Time-Lapse Imagery for Reconstructing Dynamic Objects under Occlusion](https://mlsanity.com/paper/2403.19022/) (2024)
- [Object Pose Estimation using Mid-level Visual Representations](https://mlsanity.com/paper/2203.01449/) (2022)
- [Nuisance-Label Supervision: Robustness Improvement by Free Labels](https://mlsanity.com/paper/2110.07118/) (2021)
- [Self-supervised Latent Space Optimization with Nebula Variational Coding](https://mlsanity.com/paper/2506.01414/) (2025)
- [Multi-channel Deep Supervision for Crowd Counting](https://mlsanity.com/paper/2103.09553/) (2021)
- [OOOE: Only-One-Object-Exists Assumption to Find Very Small Objects in Chest Radiographs](https://mlsanity.com/paper/2210.06806/) (2022)

## Abstract (arXiv)

Monocular 3D object parsing is highly desirable in various scenarios including occlusion reasoning and holistic scene interpretation. We present a deep convolutional neural network (CNN) architecture to localize semantic parts in 2D image and 3D space while inferring their visibility states, given a single RGB image. Our key insight is to exploit domain knowledge to regularize the network by deeply supervising its hidden layers, in order to sequentially infer intermediate concepts associated with the final task. To acquire training data in desired quantities with ground truth 3D shape and relevant concepts, we render 3D object CAD models to generate large-scale synthetic data and simulate challenging occlusion configurations between objects. We train the network only on synthetic data and demonstrate state-of-the-art performances on real image benchmarks including an extended version of KITTI, PASCAL VOC, PASCAL3D+ and IKEA for 2D and 3D keypoint localization and instance segmentation. The empirical results substantiate the utility of our deep supervision scheme by demonstrating effective transfer of knowledge from synthetic data to real images, resulting in less overfitting compared to standard end-to-end training.