# VUNet: Dynamic Scene View Synthesis for Traversability Estimation using an RGB Camera

arXiv: https://arxiv.org/abs/1806.08864 · HTML: https://mlsanity.com/paper/1806.08864/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Jun 2018.

## Built on by (most cited first, 7 of 7)

- [Navigation World Models](https://mlsanity.com/paper/2412.03572/) (2024)
- [Greedy Hierarchical Variational Autoencoders for Large-Scale Video Prediction](https://mlsanity.com/paper/2103.04174/) (2021)
- [Depth360: Self-supervised Learning for Monocular Depth Estimation using Learnable Camera Distortion Model](https://mlsanity.com/paper/2110.10415/) (2021)
- [Depth-guided Free-space Segmentation for a Mobile Robot](https://mlsanity.com/paper/2311.01966/) (2023)
- [Pushing the Limits of Learning-based Traversability Analysis for Autonomous Driving on CPU](https://mlsanity.com/paper/2206.03083/) (2022)
- [Enhancing Robustness of Indoor Robotic Navigation with Free-Space Segmentation Models Against Adversarial Attacks](https://mlsanity.com/paper/2402.08763/) (2024)
- [TravKAN: Fast and Interpretable Nonlinear Traversability Analysis with Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2608.02320/) (2026)

## Abstract (arXiv)

We present VUNet, a novel view(VU) synthesis method for mobile robots in dynamic environments, and its application to the estimation of future traversability. Our method predicts future images for given virtual robot velocity commands using only RGB images at previous and current time steps. The future images result from applying two types of image changes to the previous and current images: 1) changes caused by different camera pose, and 2) changes due to the motion of the dynamic obstacles. We learn to predict these two types of changes disjointly using two novel network architectures, SNet and DNet. We combine SNet and DNet to synthesize future images that we pass to our previously presented method GONet to estimate the traversable areas around the robot. Our quantitative and qualitative evaluation indicate that our approach for view synthesis predicts accurate future images in both static and dynamic environments. We also show that these virtual images can be used to estimate future traversability correctly. We apply our view synthesis-based traversability estimation method to two applications for assisted teleoperation.