mlsanityData to 14 Oct 2026

Representation Learning for Grounded Spatial Reasoning

arXiv 1707.03938 ↗First posted 13 Jul 2017cs.CL cs.AI cs.LGcited by 23

Not on the map. Why?

Abstract (arXiv)

The interpretation of spatial references is highly contextual, requiring joint inference over both language and the environment. We consider the task of spatial reasoning in a simulated environment, where an agent can act and receive rewards. The proposed model learns a representation of the world steered by instruction text. This design allows for precise alignment of local neighborhoods with corresponding verbalizations, while also handling global references in the instructions. We train our model with reinforcement learning using a variant of generalized value iteration. The model outperforms state-of-the-art approaches on several metrics, yielding a 45% reduction in goal localization error.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). First posted on arXiv 13 Jul 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2018: 2; 2019: 0; 2020: 0; 2021: 5; 2022: 4; 2023: 8; 2024: 1; 2025: 1; 2026: 2.

23 papers.

Sort:

By area:

  1. 2021Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotationcited by 118
  2. 2018Learning to Understand Goal Specifications by Modelling Rewardcited by 80
  3. 2018Vision-based Navigation with Language-based Assistance via Imitation Learning with Indirect Interventioncited by 77
  4. 2022StepGame: A New Benchmark for Robust Multi-Hop Spatial Reasoning in Textscited by 68
  5. 2021StructFormer: Learning Spatial Structure for Language-Guided Semantic Rearrangement of Novel Objectscited by 59
  6. 2021Grounding Language to Entities and Dynamics for Generalization in Reinforcement Learningcited by 45
  7. 2023Read and Reap the Rewards: Learning to Play Atari with the Help of Instruction Manualscited by 45
  8. 2021Grounding 'Grounding' in NLPcited by 44
  9. 2023LLF-Bench: Benchmark for Interactive Learning from Language Feedbackcited by 33
  10. 2022Learning to Execute Actions or Ask Clarification Questionscited by 30
  11. 2023Generating Visual Spatial Description via Holistic 3D Scene Understandingcited by 16
  12. 2021SILG: The Multi-environment Symbolic Interactive Language Grounding Benchmarkcited by 11
11 more
  1. 2023Planning as In-Painting: A Diffusion-Based Embodied Task Planning Framework for Environments under Uncertaintycited by 9
  2. 2023Semantically Aligned Task Decomposition in Multi-Agent Reinforcement Learningcited by 8
  3. 2023USA-Net: Unified Semantic and Affordance Representations for Robot Memorycited by 7
  4. 2023Chain-of-Symbol Prompting Elicits Planning in Large Langauge Modelscited by 6
  5. 2022Entity Divider with Language Grounding in Multi-Agent Reinforcement Learningcited by 2
  6. 2022Is the U-Net Directional-Relationship Aware?cited by 1
  7. 2026Spatial Reasoning in LLM Game Agents: Impact of Causal Context and Multi-Step Planningcited by 1
  8. 2023Solving Dialogue Grounding Embodied Task in a Simulated Environment using Further Masked Language Modelingcited by 0
  9. 2024Improving Agent Interactions in Virtual Environments with Language Modelscited by 0
  10. 2025Geometric Reasoning in the Embedding Spacecited by 0
  11. 2026TableVision: A Large-Scale Benchmark for Spatially Grounded Reasoning over Complex Hierarchical Tablescited by 0