# Spatial Imagination With Semantic Cognition for Mobile Robots

arXiv: https://arxiv.org/abs/2104.03638 · HTML: https://mlsanity.com/paper/2104.03638/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 23 references that are in the map.
- First posted on arXiv 8 Apr 2021.

## Builds on (most cited first, 17 of 17)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Dream to Control: Learning Behaviors by Latent Imagination](https://mlsanity.com/paper/1912.01603/) (2019)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [Matterport3D: Learning from RGB-D Data in Indoor Environments](https://mlsanity.com/paper/1709.06158/) (2017)
- [Habitat: A Platform for Embodied AI Research](https://mlsanity.com/paper/1904.01201/) (2019)
- [Semantic Scene Completion from a Single Depth Image](https://mlsanity.com/paper/1611.08974/) (2016)
- [Object Goal Navigation using Goal-Oriented Semantic Exploration](https://mlsanity.com/paper/2007.00643/) (2020)
- [Learning to Explore using Active Neural SLAM](https://mlsanity.com/paper/2004.05155/) (2020)
- [Imagination-Augmented Agents for Deep Reinforcement Learning](https://mlsanity.com/paper/1707.06203/) (2017)
- [SemanticFusion: Dense 3D Semantic Mapping with Convolutional Neural Networks](https://mlsanity.com/paper/1609.05130/) (2016)
- [Occupancy Anticipation for Efficient Exploration and Navigation](https://mlsanity.com/paper/2008.09285/) (2020)
- [Theta*: Any-Angle Path Planning on Grids](https://mlsanity.com/paper/1401.3843/) (2014)
- [Multi-Step Prediction of Occupancy Grid Maps with Recurrent Neural Networks](https://mlsanity.com/paper/1812.09395/) (2018)

## Built on by (most cited first, 3 of 3)

- [Navigating to Objects in Unseen Environments by Distance Prediction](https://mlsanity.com/paper/2202.03735/) (2022)
- [4CNet: A Diffusion Approach to Map Prediction for Decentralized Multi-Robot Exploration](https://mlsanity.com/paper/2402.17904/) (2024)
- [VSGM -- Enhance robot task understanding ability through visual semantic graph](https://mlsanity.com/paper/2105.08959/) (2021)

## Abstract (arXiv)

The imagination of the surrounding environment based on experience and semantic cognition has great potential to extend the limited observations and provide more information for mapping, collision avoidance, and path planning. This paper provides a training-based algorithm for mobile robots to perform spatial imagination based on semantic cognition and evaluates the proposed method for the mapping task. We utilize a photo-realistic simulation environment, Habitat, for training and evaluation. The trained model is composed of Resent-18 as encoder and Unet as the backbone. We demonstrate that the algorithm can perform imagination for unseen parts of the object universally, by recalling the images and experience and compare our approach with traditional semantic mapping methods. It is found that our approach will improve the efficiency and accuracy of semantic mapping.