# Bounding Boxes as Goals: Language-Conditioned Grasping via Neuro-Symbolic Planning

arXiv: https://arxiv.org/abs/2606.12910 · HTML: https://mlsanity.com/paper/2606.12910/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 31 references that are in the map.
- First posted on arXiv 11 Jun 2026.

## Builds on (most cited first, 22 of 22)

- [OpenAI GPT-5 System Card](https://mlsanity.com/paper/2601.03267/) (2025)
- [CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks](https://mlsanity.com/paper/2112.03227/) (2021)
- [CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulation](https://mlsanity.com/paper/2411.19650/) (2024)
- [VLABench: A Large-Scale Benchmark for Language-Conditioned Robotics Manipulation with Long-Horizon Reasoning Tasks](https://mlsanity.com/paper/2412.18194/) (2024)
- [Open-World Object Manipulation using Pre-trained Vision-Language Models](https://mlsanity.com/paper/2303.00905/) (2023)
- [Large Language Models for Robotics: A Survey](https://mlsanity.com/paper/2311.07226/) (2023)
- [RoboMamba: Efficient Vision-Language-Action Model for Robotic Reasoning and Manipulation](https://mlsanity.com/paper/2406.04339/) (2024)
- [Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Survey](https://mlsanity.com/paper/2508.13073/) (2025)
- [RoboGround: Robotic Manipulation with Grounded Vision-Language Priors](https://mlsanity.com/paper/2504.21530/) (2025)
- [Vision Language Action Models in Robotic Manipulation: A Systematic Review](https://mlsanity.com/paper/2507.10672/) (2025)
- [Learning Visual Grounding from Generative Vision and Language Model](https://mlsanity.com/paper/2407.14563/) (2024)
- [A Survey of Robotic Language Grounding: Tradeoffs between Symbols and Embeddings](https://mlsanity.com/paper/2405.13245/) (2024)
- [A Survey of Optimization-based Task and Motion Planning: From Classical To Learning Approaches](https://mlsanity.com/paper/2404.02817/) (2024)
- [Neuro-Symbolic Artificial Intelligence: Towards Improving the Reasoning Abilities of Large Language Models](https://mlsanity.com/paper/2508.13678/) (2025)
- [LoHoRavens: A Long-Horizon Language-Conditioned Benchmark for Robotic Tabletop Manipulation](https://mlsanity.com/paper/2310.12020/) (2023)
- [Learning Efficient and Robust Language-conditioned Manipulation using Textual-Visual Relevancy and Equivariant Language Mapping](https://mlsanity.com/paper/2406.15677/) (2024)
- [LEMMA: Learning Language-Conditioned Multi-Robot Manipulation](https://mlsanity.com/paper/2308.00937/) (2023)
- [ESCA: Contextualizing Embodied Agents via Scene-Graph Generation](https://mlsanity.com/paper/2510.15963/) (2025)
- [Domain-Conditioned Scene Graphs for State-Grounded Task Planning](https://mlsanity.com/paper/2504.06661/) (2025)
- [Versatile and Generalizable Manipulation via Goal-Conditioned Reinforcement Learning with Grounded Object Detection](https://mlsanity.com/paper/2507.10814/) (2025)
- [Structured Interfaces for Automated Reasoning with 3D Scene Graphs](https://mlsanity.com/paper/2510.16643/) (2025)
- [Improving Generalization of Language-Conditioned Robot Manipulation](https://mlsanity.com/paper/2508.02405/) (2025)

## Abstract (arXiv)

For robotics to be effectively integrated into household or industrial environments, machines must adapt to natural-language prompts in real time. Although Vision-Language Models (VLMs) have enabled zero-shot generalization in robot task and motion planning (TAMP), current state-of-the-art approaches often remain computationally "heavyweight" or require extensive training on thousands of demonstrations. We present GRASP (Grounded Reasoning and Symbolic Planning), a framework designed as a step toward open-vocabulary tabletop manipulation. Our approach leverages a pretrained VLM to translate natural-language queries into neuro-symbolic goal states, grounded in the physical world via a bounding-box detection pipeline. Unlike methods that rely on fixed color lists or hard-coded coordinates, GRASP enables robots to interpret abstract spatial concepts such as "top shelf" and execute tasks without additional fine-tuning. We achieve 73.3% overall success across 90 real-robot trials at three difficulty levels, requiring no task-specific training.