# Deep Affordance-grounded Sensorimotor Object Recognition

arXiv: https://arxiv.org/abs/1704.02787 · HTML: https://mlsanity.com/paper/1704.02787/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Apr 2017.

## Built on by (most cited first, 17 of 17)

- [AffordPose: A Large-scale Dataset of Hand-Object Interactions with Affordance-driven Hand Pose](https://mlsanity.com/paper/2309.08942/) (2023)
- [Open-Vocabulary Affordance Detection in 3D Point Clouds](https://mlsanity.com/paper/2303.02401/) (2023)
- [One-Shot Object Affordance Detection in the Wild](https://mlsanity.com/paper/2108.03658/) (2021)
- [Multimodal Explainable Artificial Intelligence: A Comprehensive Review of Methodological Advances and Future Research Directions](https://mlsanity.com/paper/2306.05731/) (2023)
- [Phrase-Based Affordance Detection via Cyclic Bilateral Interaction](https://mlsanity.com/paper/2202.12076/) (2022)
- [Fine-grained Affordance Annotation for Egocentric Hand-Object Interaction Videos](https://mlsanity.com/paper/2302.03292/) (2023)
- [Bayesian Deep Learning for Affordance Segmentation in images](https://mlsanity.com/paper/2303.00871/) (2023)
- [Beyond Object Recognition: A New Benchmark towards Object Concept Learning](https://mlsanity.com/paper/2212.02710/) (2022)
- [User profile-driven large-scale multi-agent learning from demonstration in federated human-robot collaborative environments](https://mlsanity.com/paper/2103.16434/) (2021)
- [Building Affordance Relations for Robotic Agents - A Review](https://mlsanity.com/paper/2105.06706/) (2021)
- [Illicit item detection in X-ray images for security applications](https://mlsanity.com/paper/2305.01936/) (2023)
- [Visual inspection for illicit items in X-ray images using Deep Learning](https://mlsanity.com/paper/2310.03658/) (2023)
- [DeepMoCap: Deep Optical Motion Capture Using Multiple Depth Sensors and Retro-Reflectors](https://mlsanity.com/paper/2110.07283/) (2021)
- [Precise Affordance Annotation for Egocentric Action Video Datasets](https://mlsanity.com/paper/2206.05424/) (2022)
- [Learning Scene Flow With Skeleton Guidance For 3D Action Recognition](https://mlsanity.com/paper/2306.13285/) (2023)
- [Capturing Fine-Grained Alignments Improves 3D Affordance Detection](https://mlsanity.com/paper/2506.19312/) (2025)
- [YOLOA: Real-Time Affordance Detection via LLM Adapter](https://mlsanity.com/paper/2512.03418/) (2025)

## Abstract (arXiv)

It is well-established by cognitive neuroscience that human perception of objects constitutes a complex process, where object appearance information is combined with evidence about the so-called object "affordances", namely the types of actions that humans typically perform when interacting with them. This fact has recently motivated the "sensorimotor" approach to the challenging task of automatic object recognition, where both information sources are fused to improve robustness. In this work, the aforementioned paradigm is adopted, surpassing current limitations of sensorimotor object recognition research. Specifically, the deep learning paradigm is introduced to the problem for the first time, developing a number of novel neuro-biologically and neuro-physiologically inspired architectures that utilize state-of-the-art neural networks for fusing the available information sources in multiple ways. The proposed methods are evaluated using a large RGB-D corpus, which is specifically collected for the task of sensorimotor object recognition and is made publicly available. Experimental results demonstrate the utility of affordance information to object recognition, achieving an up to 29% relative error reduction by its inclusion.