# CRAFT: A Neuro-Symbolic Framework for Visual Functional Affordance Grounding

arXiv: https://arxiv.org/abs/2507.14426 · HTML: https://mlsanity.com/paper/2507.14426/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 16 references that are in the map.
- First posted on arXiv 19 Jul 2025.

## Builds on (most cited first, 10 of 10)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [ConceptNet 5.5: An Open Multilingual Graph of General Knowledge](https://mlsanity.com/paper/1612.03975/) (2016)
- [CoTDet: Affordance Knowledge Prompting for Task Driven Object Detection](https://mlsanity.com/paper/2309.01093/) (2023)
- [Robot Object Retrieval with Contextual Natural Language Queries](https://mlsanity.com/paper/2006.13253/) (2020)
- [What Object Should I Use? - Task Driven Object Detection](https://mlsanity.com/paper/1904.03000/) (2019)
- [TaskCLIP: Extend Large Vision-Language Model for Task Oriented Object Detection](https://mlsanity.com/paper/2403.08108/) (2024)
- [What does CLIP know about peeling a banana?](https://mlsanity.com/paper/2404.12015/) (2024)
- [Discovering Novel Actions from Open World Egocentric Videos with Object-Grounded Visual Commonsense Reasoning](https://mlsanity.com/paper/2305.16602/) (2023)
- [Discovering Object Attributes by Prompting Large Language Models with Perception-Action APIs](https://mlsanity.com/paper/2409.15505/) (2024)

## Built on by (most cited first, 1 of 1)

- [CRAFT-E: A Neuro-Symbolic Framework for Embodied Affordance Grounding](https://mlsanity.com/paper/2512.04231/) (2025)

## Abstract (arXiv)

We introduce CRAFT, a neuro-symbolic framework for interpretable affordance grounding, which identifies the objects in a scene that enable a given action (e.g., "cut"). CRAFT integrates structured commonsense priors from ConceptNet and language models with visual evidence from CLIP, using an energy-based reasoning loop to refine predictions iteratively. This process yields transparent, goal-driven decisions to ground symbolic and perceptual structures. Experiments in multi-object, label-free settings demonstrate that CRAFT enhances accuracy while improving interpretability, providing a step toward robust and trustworthy scene understanding.