# Footprints and Free Space from a Single Color Image

arXiv: https://arxiv.org/abs/2004.06376 · HTML: https://mlsanity.com/paper/2004.06376/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Apr 2020.

## Built on by (most cited first, 6 of 6)

- [Sparse Auxiliary Networks for Unified Monocular Depth Prediction and Completion](https://mlsanity.com/paper/2103.16690/) (2021)
- [Trans4Trans: Efficient Transformer for Transparent Object and Semantic Scene Segmentation in Real-World Navigation Assistance](https://mlsanity.com/paper/2108.09174/) (2021)
- [HIDA: Towards Holistic Indoor Understanding for the Visually Impaired via Semantic Instance Segmentation with a Wearable Solid-State LiDAR Sensor](https://mlsanity.com/paper/2107.03180/) (2021)
- [Panoptic Lintention Network: Towards Efficient Navigational Perception for the Visually Impaired](https://mlsanity.com/paper/2103.04128/) (2021)
- [FusionForce: End-to-end Differentiable Neural-Symbolic Layer for Trajectory Prediction](https://mlsanity.com/paper/2502.10156/) (2025)
- [BlindSpotNet: Seeing Where We Cannot See](https://mlsanity.com/paper/2207.03870/) (2022)

## Abstract (arXiv)

Understanding the shape of a scene from a single color image is a formidable computer vision task. However, most methods aim to predict the geometry of surfaces that are visible to the camera, which is of limited use when planning paths for robots or augmented reality agents. Such agents can only move when grounded on a traversable surface, which we define as the set of classes which humans can also walk over, such as grass, footpaths and pavement. Models which predict beyond the line of sight often parameterize the scene with voxels or meshes, which can be expensive to use in machine learning frameworks. We introduce a model to predict the geometry of both visible and occluded traversable surfaces, given a single RGB image as input. We learn from stereo video sequences, using camera poses, per-frame depth and semantic segmentation to form training data, which is used to supervise an image-to-image network. We train models from the KITTI driving dataset, the indoor Matterport dataset, and from our own casually captured stereo footage. We find that a surprisingly low bar for spatial coverage of training scenes is required. We validate our algorithm against a range of strong baselines, and include an assessment of our predictions for a path-planning task.