# LINGO-Space: Language-Conditioned Incremental Grounding for Space

arXiv: https://arxiv.org/abs/2402.01183 · HTML: https://mlsanity.com/paper/2402.01183/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 42 references that are in the map.
- First posted on arXiv 2 Feb 2024.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [Code as Policies: Language Model Programs for Embodied Control](https://mlsanity.com/paper/2209.07753/) (2022)
- [Inner Monologue: Embodied Reasoning through Planning with Language Models](https://mlsanity.com/paper/2207.05608/) (2022)
- [Strategies for Pre-training Graph Neural Networks](https://mlsanity.com/paper/1905.12265/) (2019)
- [Recipe for a General, Powerful, Scalable Graph Transformer](https://mlsanity.com/paper/2205.12454/) (2022)
- [CLIPort: What and Where Pathways for Robotic Manipulation](https://mlsanity.com/paper/2109.12098/) (2021)
- [Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Items](https://mlsanity.com/paper/2204.11918/) (2022)
- [LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action](https://mlsanity.com/paper/2207.04429/) (2022)
- [Transporter Networks: Rearranging the Visual World for Robotic Manipulation](https://mlsanity.com/paper/2010.14406/) (2020)
- [Language to Logical Form with Neural Attention](https://mlsanity.com/paper/1601.01280/) (2016)
- [Bottom Up Top Down Detection Transformers for Language Grounding in Images and Point Clouds](https://mlsanity.com/paper/2112.08879/) (2021)
- [Interactively Picking Real-World Objects with Unconstrained Spoken Language Instructions](https://mlsanity.com/paper/1710.06280/) (2017)
- [Energy-based Models are Zero-Shot Planners for Compositional Scene Rearrangement](https://mlsanity.com/paper/2304.14391/) (2023)
- [Leveraging Language for Accelerated Learning of Tool Manipulation](https://mlsanity.com/paper/2206.13074/) (2022)
- [Predicting Stable Configurations for Semantic Placement of Novel Objects](https://mlsanity.com/paper/2108.12062/) (2021)
- [Ground then Navigate: Language-guided Navigation in Dynamic Scenes](https://mlsanity.com/paper/2209.11972/) (2022)
- [Learning Object Placements For Relational Instructions by Hallucinating Scene Representations](https://mlsanity.com/paper/2001.08481/) (2020)
- [Spatial Reasoning from Natural Language Instructions for Robot Manipulation](https://mlsanity.com/paper/2012.13693/) (2020)
- [Learning Neuro-symbolic Programs for Language Guided Robot Manipulation](https://mlsanity.com/paper/2211.06652/) (2022)
- [Differentiable Parsing and Visual Grounding of Natural Language Instructions for Object Placement](https://mlsanity.com/paper/2210.00215/) (2022)

## Built on by (most cited first, 8 of 8)

- [QueSTMaps: Queryable Semantic Topological Maps for 3D Scene Understanding](https://mlsanity.com/paper/2404.06442/) (2024)
- [Sketch-Plan-Generalize: Learning and Planning with Neuro-Symbolic Programmatic Representations for Inductive Spatial Concepts](https://mlsanity.com/paper/2404.07774/) (2024)
- [Neuro-LIFT: A Neuromorphic, LLM-based Interactive Framework for Autonomous Drone FlighT at the Edge](https://mlsanity.com/paper/2501.19259/) (2025)
- [C2F-Space: Coarse-to-Fine Space Grounding for Spatial Instructions using Vision-Language Models](https://mlsanity.com/paper/2511.15333/) (2025)
- [BEACON: Language-Conditioned Navigation Affordance Prediction under Occlusion](https://mlsanity.com/paper/2603.09961/) (2026)
- [Graph-Enhanced Large Language Models for Spatial Search](https://mlsanity.com/paper/2606.22909/) (2026)
- [ReRef-3D: A Benchmark for Spatial Referring Expression-Guided 3D Scene Rearrangement](https://mlsanity.com/paper/2608.16011/) (2026)
- [MIND: Instruction-Conditioned Incremental Grounding for Robots](https://mlsanity.com/paper/2610.19262/) (2026)

## Abstract (arXiv)

We aim to solve the problem of spatially localizing composite instructions referring to space: space grounding. Compared to current instance grounding, space grounding is challenging due to the ill-posedness of identifying locations referred to by discrete expressions and the compositional ambiguity of referring expressions. Therefore, we propose a novel probabilistic space-grounding methodology (LINGO-Space) that accurately identifies a probabilistic distribution of space being referred to and incrementally updates it, given subsequent referring expressions leveraging configurable polar distributions. Our evaluations show that the estimation using polar distributions enables a robot to ground locations successfully through 20 table-top manipulation benchmark tests. We also show that updating the distribution helps the grounding method accurately narrow the referring space. We finally demonstrate the robustness of the space grounding with simulated manipulation and real quadruped robot navigation tasks. Code and videos are available at https://lingo-space.github.io.