mlsanityData to 7 Oct 2026

Learning Interpretable Spatial Operations in a Rich 3D Blocks World

arXiv 1712.03463 ↗First posted 10 Dec 2017cs.CLcited by 15

Not on the map. Why?

Abstract (arXiv)

In this paper, we study the problem of mapping natural language instructions to complex spatial actions in a 3D blocks world. We first introduce a new dataset that pairs complex 3D spatial operations to rich natural language descriptions that require complex spatial and pragmatic interpretations such as "mirroring", "twisting", and "balancing". This dataset, built on the simulation environment of Bisk, Yuret, and Marcu (2016), attains language that is significantly richer and more complex, while also doubling the size of the original dataset in the 2D environment with 100 new world configurations and 250,000 tokens. In addition, we propose a new neural architecture that achieves competitive results while automatically discovering an inventory of interpretable spatial operations (Figure 5)

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). First posted on arXiv 10 Dec 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 1; 2019: 0; 2020: 0; 2021: 5; 2022: 5; 2023: 2; 2024: 1; 2025: 0; 2026: 1.

15 papers.

Sort:

By area:

  1. 2022Interactive Language: Talking to Robots in Real Timecited by 217
  2. 2021TEACh: Task-driven Embodied Agents that Chatcited by 194
  3. 2018Vision-based Navigation with Language-based Assistance via Imitation Learning with Indirect Interventioncited by 77
  4. 2022StepGame: A New Benchmark for Robust Multi-Hop Spatial Reasoning in Textscited by 65
  5. 2021MindCraft: Theory of Mind Modeling for Situated Dialogue in Collaborative Taskscited by 62
  6. 2021StructFormer: Learning Spatial Structure for Language-Guided Semantic Rearrangement of Novel Objectscited by 57
  7. 2021Draw Me a Flower: Processing and Grounding Abstraction in Natural Languagecited by 14
  8. 2021FixMyPose: Pose Correctional Captioning and Retrievalcited by 12
  9. 2022Reasoning about Actions over Visual and Linguistic Modalities: A Surveycited by 10
  10. 2022Integrating AI Planning with Natural Language Processing: A Combination of Explicit and Tacit Knowledgecited by 3
  11. 2024ActionCOMET: A Zero-shot Approach to Learn Image-specific Commonsense Concepts about Actionscited by 2
  12. 2022On the Limits of Evaluating Embodied Agent Model Generalization Using Validation Setscited by 1
3 more
  1. 2023Interaction is all You Need? A Study of Robots Ability to Understand and Executecited by 1
  2. 2026Distributed Partial Information Puzzles: Examining Common Ground Construction Under Epistemic Asymmetrycited by 1
  3. 2023Emergent Communication for Rules Reasoningcited by 0