# Inferring Compact Representations for Efficient Natural Language Understanding of Robot Instructions

arXiv: https://arxiv.org/abs/1903.09243 · HTML: https://mlsanity.com/paper/1903.09243/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Mar 2019.

## Built on by (most cited first, 9 of 9)

- [Hi Robot: Open-Ended Instruction Following with Hierarchical Vision-Language-Action Models](https://mlsanity.com/paper/2502.19417/) (2025)
- [A Persistent Spatial Semantic Representation for High-level Natural Language Instruction Execution](https://mlsanity.com/paper/2107.05612/) (2021)
- [Chat with the Environment: Interactive Multimodal Perception Using Large Language Models](https://mlsanity.com/paper/2303.08268/) (2023)
- [From Machine Learning to Robotics: Challenges and Opportunities for Embodied Intelligence](https://mlsanity.com/paper/2110.15245/) (2021)
- [Robix: A Unified Model for Robot Interaction, Reasoning and Planning](https://mlsanity.com/paper/2509.01106/) (2025)
- [Transcrib3D: 3D Referring Expression Resolution through Large Language Models](https://mlsanity.com/paper/2404.19221/) (2024)
- [Embodied Concept Learner: Self-supervised Learning of Concepts and Mapping through Instruction Following](https://mlsanity.com/paper/2304.03767/) (2023)
- [Language Understanding for Field and Service Robots in a Priori Unknown Environments](https://mlsanity.com/paper/2105.10396/) (2021)
- [Active Exploration based on Information Gain by Particle Filter for Efficient Spatial Concept Formation](https://mlsanity.com/paper/2211.10934/) (2022)

## Abstract (arXiv)

The speed and accuracy with which robots are able to interpret natural language is fundamental to realizing effective human-robot interaction. A great deal of attention has been paid to developing models and approximate inference algorithms that improve the efficiency of language understanding. However, existing methods still attempt to reason over a representation of the environment that is flat and unnecessarily detailed, which limits scalability. An open problem is then to develop methods capable of producing the most compact environment model sufficient for accurate and efficient natural language understanding. We propose a model that leverages environment-related information encoded within instructions to identify the subset of observations and perceptual classifiers necessary to perceive a succinct, instruction-specific environment representation. The framework uses three probabilistic graphical models trained from a corpus of annotated instructions to infer salient scene semantics, perceptual classifiers, and grounded symbols. Experimental results on two robots operating in different environments demonstrate that by exploiting the content and the structure of the instructions, our method learns compact environment representations that significantly improve the efficiency of natural language symbol grounding.