# Domain-Conditioned Scene Graphs for State-Grounded Task Planning

arXiv: https://arxiv.org/abs/2504.06661 · HTML: https://mlsanity.com/paper/2504.06661/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- First posted on arXiv 9 Apr 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)

## Built on by (most cited first, 4 of 4)

- [Vision Language Models Cannot Plan, but Can They Formalize?](https://mlsanity.com/paper/2509.21576/) (2025)
- [Learning Hierarchical Domain Models Through Environment-Grounded Interaction](https://mlsanity.com/paper/2505.13497/) (2025)
- [TaskGround: Structured Executable Task Inference for Full-Scene Household Reasoning](https://mlsanity.com/paper/2605.18109/) (2026)
- [Bounding Boxes as Goals: Language-Conditioned Grasping via Neuro-Symbolic Planning](https://mlsanity.com/paper/2606.12910/) (2026)

## Abstract (arXiv)

Recent robotic task planning frameworks have integrated large multimodal models (LMMs) such as GPT-4o. To address grounding issues of such models, it has been suggested to split the pipeline into perceptional state grounding and subsequent state-based planning. As we show in this work, the state grounding ability of LMM-based approaches is still limited by weaknesses in granular, structured, domain-specific scene understanding. To address this shortcoming, we develop a more structured state grounding framework that features a domain-conditioned scene graph as its scene representation. We show that such representation is actionable in nature as it is directly mappable to a symbolic state in planning languages such as the Planning Domain Definition Language (PDDL). We provide an instantiation of our state grounding framework where the domain-conditioned scene graph generation is implemented with a lightweight vision-language approach that classifies domain-specific predicates on top of domain-relevant object detections. Evaluated across three domains, our approach achieves significantly higher state rounding accuracy and task planning success rates compared to LMM-based approaches.