# Hallucinating Beyond Observation: Learning to Complete with Partial Observation and Unpaired Prior Knowledge

arXiv: https://arxiv.org/abs/1907.09786 · HTML: https://mlsanity.com/paper/1907.09786/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Jul 2019.

## Built on by (most cited first, 2 of 2)

- [Grid-Centric Traffic Scenario Perception for Autonomous Driving: A Comprehensive Review](https://mlsanity.com/paper/2303.01212/) (2023)
- [AutoLay: Benchmarking amodal layout estimation for autonomous driving](https://mlsanity.com/paper/2108.09047/) (2021)

## Abstract (arXiv)

We propose a novel single-step training strategy that allows convolutional encoder-decoder networks that use skip connections, to complete partially observed data by means of hallucination. This strategy is demonstrated for the task of completing 2-D road layouts as well as 3-D vehicle shapes. As input, it takes data from a partially observed domain, for which no ground truth is available, and data from an unpaired prior knowledge domain and trains the network in an end-to-end manner. Our single-step training strategy is compared against two state-of-the-art baselines, one using a two-step auto-encoder training strategy and one using an adversarial strategy. Our novel strategy achieves an improvement up to +12.2% F-measure on the Cityscapes dataset. The learned network intrinsically generalizes better than the baselines on unseen datasets, which is demonstrated by an improvement up to +23.8% F-measure on the unseen KITTI dataset. Moreover, our approach outperforms the baselines using the same backbone network on the 3-D shape completion benchmark by a margin of 0.006 Hamming distance.