Hierarchical Path-planning from Speech Instructions with Spatial Concept-based Topometric Semantic Mapping
Not on the map. Why?
Abstract (arXiv)
Assisting individuals in their daily activities through autonomous mobile robots, especially for users without specialized knowledge, is crucial. Specifically, the capability of robots to navigate to destinations based on human speech instructions is essential. While robots can take different paths to the same goal, the shortest path is not always the best. A preferred approach is to accommodate waypoint specifications flexibly, planning an improved alternative path, even with detours. Additionally, robots require real-time inference capabilities. This study aimed to realize a hierarchical spatial representation using a topometric semantic map and path planning with speech instructions, including waypoints. This paper presents Spatial Concept-based Topometric Semantic Mapping for Hierarchical Path Planning (SpCoTMHP), integrating place connectivity. This approach offers a novel integrated probabilistic generative model and fast approximate inference across hierarchy levels. A formulation based on control as probabilistic inference theoretically supports the proposed path planning algorithm. We conducted experiments in home environments using the Toyota Human Support Robot on the SIGVerse simulator and in a lab-office environment with the real robot, Albert. Users issued speech commands specifying the waypoint and goal, such as "Go to the bedroom via the corridor." Navigation experiments using speech instructions with a waypoint demonstrated a performance improvement of SpCoTMHP over the baseline hierarchical path planning method with heuristic path costs (HPP-I), in terms of the weighted success rate at which the robot reaches the closest target and passes the correct waypoints, by 0.590. The computation time was significantly accelerated by 7.14 seconds with SpCoTMHP compared to baseline HPP-I in advanced tasks.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 24 of its 82 references that are in the map. First posted on arXiv 21 Mar 2022.
Builds on
24 of 82 references.
References · 24
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
- 2021ClipCap: CLIP Prefix for Image Captioningcited by 559
- 2018On Evaluation of Embodied Navigation Agentscited by 547
- 2018Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Reviewcited by 482
- 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 441
- 2016Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivationcited by 286
- 2021Kimera: from SLAM to Spatial Perception with 3D Dynamic Scene Graphscited by 144
- 2020Topological Planning with Transformers for Vision-and-Language Navigationcited by 116
- 2018Latent Space Policies for Hierarchical Reinforcement Learningcited by 67
- 2021Semantics for Robotic Mapping, Perception and Interaction: A Surveycited by 56
12 more
- 2013Rao-Blackwellised Particle Filtering for Dynamic Bayesian Networkscited by 47
- 2018Symbol Emergence in Cognitive Developmental Systems: a Surveycited by 39
- 2015Symbol Emergence in Robotics: A Surveycited by 31
- 2019Neuro-SERKET: Development of Integrative Cognitive System through the Composition of Deep Probabilistic Generative Modelscited by 20
- 2018Improved and Scalable Online Learning of Spatial Concepts and Language Models with Mappingcited by 13
- 2017Online Spatial Concept and Lexical Acquisition with Simultaneous Localization and Mappingcited by 10
- 2020Spatial Concept-Based Navigation with Human Speech Instructions via Probabilistic Inference on Bayesian Generative Modelcited by 9
- 2019Integration of Imitation Learning using GAIL and Reinforcement Learning using Task-achievement Rewards via Probabilistic Graphical Modelcited by 7
- 2016Spatial Concept Acquisition for a Mobile Robot that Integrates Self-Localization and Unsupervised Word Discovery from Spoken Sentencescited by 6
- 2020Learning Topometric Semantic Maps from Occupancy Gridscited by 2
- 2020Enabling Topological Planning with Monocular Visioncited by 2
- 2021Topological Semantic Mapping by Consolidation of Deep Visual Featurescited by 1
Built on by
Citing papers per year: 2025: 1.
1 paper.