# Weakly But Deeply Supervised Occlusion-Reasoned Parametric Road Layouts

arXiv: https://arxiv.org/abs/2104.06730 · HTML: https://mlsanity.com/paper/2104.06730/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 51 references that are in the map.
- First posted on arXiv 14 Apr 2021.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [Two-Stream Convolutional Networks for Action Recognition in Videos](https://mlsanity.com/paper/1406.2199/) (2014)
- [Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3D](https://mlsanity.com/paper/2008.05711/) (2020)
- [Unsupervised Monocular Depth Estimation with Left-Right Consistency](https://mlsanity.com/paper/1609.03677/) (2016)
- [Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs](https://mlsanity.com/paper/1412.7062/) (2014)
- [Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving](https://mlsanity.com/paper/1812.07179/) (2018)
- [Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue](https://mlsanity.com/paper/1603.04992/) (2016)
- [Deeply-Supervised Nets](https://mlsanity.com/paper/1409.5185/) (2014)
- [Cognitive Mapping and Planning for Visual Navigation](https://mlsanity.com/paper/1702.03920/) (2017)
- [Orthographic Feature Transform for Monocular 3D Object Detection](https://mlsanity.com/paper/1811.08188/) (2018)
- [Deep Feature Flow for Video Recognition](https://mlsanity.com/paper/1611.07715/) (2016)
- [Flow-Guided Feature Aggregation for Video Object Detection](https://mlsanity.com/paper/1703.10025/) (2017)
- [Predicting Semantic Map Representations from Images using Pyramid Occupancy Networks](https://mlsanity.com/paper/2003.13402/) (2020)
- [Cross-view Semantic Segmentation for Sensing Surroundings](https://mlsanity.com/paper/1906.03560/) (2019)
- [Stereo R-CNN based 3D Object Detection for Autonomous Driving](https://mlsanity.com/paper/1902.09738/) (2019)
- [Self-Supervised Scene De-occlusion](https://mlsanity.com/paper/2004.02788/) (2020)
- [Monocular Semantic Occupancy Grid Mapping with Convolutional Variational Encoder-Decoder Networks](https://mlsanity.com/paper/1804.02176/) (2018)
- [Layer-structured 3D Scene Inference via View Synthesis](https://mlsanity.com/paper/1807.10264/) (2018)
- [MonoLayout: Amodal scene layout from a single image](https://mlsanity.com/paper/2002.08394/) (2020)
- [Learning to Look around Objects for Top-View Representations of Outdoor Scenes](https://mlsanity.com/paper/1803.10870/) (2018)
- [Object-Driven Multi-Layer Scene Decomposition From a Single Image](https://mlsanity.com/paper/1908.09521/) (2019)
- [Im2Pano3D: Extrapolating 360 Structure and Semantics Beyond the Field of View](https://mlsanity.com/paper/1712.04569/) (2017)
- [A Parametric Top-View Representation of Complex Road Scenes](https://mlsanity.com/paper/1812.06152/) (2018)
- [Deep Supervision with Shape Concepts for Occlusion-Aware 3D Object Parsing](https://mlsanity.com/paper/1612.02699/) (2016)
- [Understanding Road Layout from Videos as a Whole](https://mlsanity.com/paper/2007.00822/) (2020)
- [Learning from Maps: Visual Common Sense for Autonomous Driving](https://mlsanity.com/paper/1611.08583/) (2016)
- [Memory Warps for Learning Long-Term Online Video Representations](https://mlsanity.com/paper/1803.10861/) (2018)

## Built on by (most cited first, 3 of 3)

- [Image Amodal Completion: A Survey](https://mlsanity.com/paper/2207.02062/) (2022)
- [Object-level Scene Deocclusion](https://mlsanity.com/paper/2406.07706/) (2024)
- [MVPbev: Multi-view Perspective Image Generation from BEV with Test-time Controllability and Generalizability](https://mlsanity.com/paper/2407.19468/) (2024)

## Abstract (arXiv)

We propose an end-to-end network that takes a single perspective RGB image of a complex road scene as input, to produce occlusion-reasoned layouts in perspective space as well as a parametric bird's-eye-view (BEV) space. In contrast to prior works that require dense supervision such as semantic labels in perspective view, our method only requires human annotations for parametric attributes that are cheaper and less ambiguous to obtain. To solve this challenging task, our design is comprised of modules that incorporate inductive biases to learn occlusion-reasoning, geometric transformation and semantic abstraction, where each module may be supervised by appropriately transforming the parametric annotations. We demonstrate how our design choices and proposed deep supervision help achieve meaningful representations and accurate predictions. We validate our approach on two public datasets, KITTI and NuScenes, to achieve state-of-the-art results with considerably less human supervision.