# Grounding Object Relations in Language-Conditioned Robotic Manipulation with Semantic-Spatial Reasoning

arXiv: https://arxiv.org/abs/2303.17919 · HTML: https://mlsanity.com/paper/2303.17919/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning.
- Builds on 0 of its 28 references that are in the map.
- First posted on arXiv 31 Mar 2023.

## Placed in

- LLM reasoning / [Spatial and embodied reasoning](https://mlsanity.com/lr/spatial-and-embodied-reasoning/)

## Built on by (most cited first, 4 of 4)

- [Learning 3D Object Spatial Relationships from Pre-trained 2D Diffusion Models](https://mlsanity.com/paper/2503.19914/) (2025)
- [DepWiGNN: A Depth-wise Graph Neural Network for Multi-hop Spatial Reasoning in Text](https://mlsanity.com/paper/2310.12557/) (2023)
- [BBSEA: An Exploration of Brain-Body Synchronization for Embodied Agents](https://mlsanity.com/paper/2402.08212/) (2024)
- [SafeRelBench: A Spatial-Relation-Aware Benchmark for Process-Level Safety in VLM-Driven Embodied Agents](https://mlsanity.com/paper/2607.14543/) (2026)

## Abstract (arXiv)

Grounded understanding of natural language in physical scenes can greatly benefit robots that follow human instructions. In object manipulation scenarios, existing end-to-end models are proficient at understanding semantic concepts, but typically cannot handle complex instructions involving spatial relations among multiple objects. which require both reasoning object-level spatial relations and learning precise pixel-level manipulation affordances. We take an initial step to this challenge with a decoupled two-stage solution. In the first stage, we propose an object-centric semantic-spatial reasoner to select which objects are relevant for the language instructed task. The segmentation of selected objects are then fused as additional input to the affordance learning stage. Simply incorporating the inductive bias of relevant objects to a vision-language affordance learning agent can effectively boost its performance in a custom testbed designed for object manipulation with spatial-related language instructions.