# Building Affordance Relations for Robotic Agents - A Review

arXiv: https://arxiv.org/abs/2105.06706 · HTML: https://mlsanity.com/paper/2105.06706/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 124 references that are in the map.
- First posted on arXiv 14 May 2021.

## Builds on (most cited first, 12 of 12)

- [AffordanceNet: An End-to-End Deep Learning Approach for Object Affordance Detection](https://mlsanity.com/paper/1709.07326/) (2017)
- [Interactive Perception: Leveraging Action in Perception and Perception in Action](https://mlsanity.com/paper/1604.03670/) (2016)
- [SE3-Nets: Learning Rigid Body Motion using Deep Neural Networks](https://mlsanity.com/paper/1606.02378/) (2016)
- [Learning Grasp Affordance Reasoning through Semantic Relations](https://mlsanity.com/paper/1906.09836/) (2019)
- [Learning Social Affordance for Human-Robot Interaction](https://mlsanity.com/paper/1604.03692/) (2016)
- [Deep Affordance-grounded Sensorimotor Object Recognition](https://mlsanity.com/paper/1704.02787/) (2017)
- [RoboBrain: Large-Scale Knowledge Engine for Robots](https://mlsanity.com/paper/1412.0691/) (2014)
- [What Can I Do Around Here? Deep Functional Scene Understanding for Cognitive Robots](https://mlsanity.com/paper/1602.00032/) (2016)
- [Affordance-Aware Handovers with Human Arm Mobility Constraints](https://mlsanity.com/paper/2010.15436/) (2020)
- [MetaGrasp: Data Efficient Grasping by Affordance Interpreter Network](https://mlsanity.com/paper/1902.06554/) (2019)
- [Learning Social Affordance Grammar from Videos: Transferring Human Interactions to Human-Robot Interactions](https://mlsanity.com/paper/1703.00503/) (2017)
- [Self-Assessment of Grasp Affordance Transfer](https://mlsanity.com/paper/2007.02132/) (2020)

## Built on by (most cited first, 7 of 7)

- [AffordBot: 3D Fine-grained Embodied Reasoning via Multimodal Large Language Models](https://mlsanity.com/paper/2511.10017/) (2025)
- [Recent Advances of Deep Robotic Affordance Learning: A Reinforcement Learning Perspective](https://mlsanity.com/paper/2303.05344/) (2023)
- [Scaling Single Human Demonstrations for Imitation Learning using Generative Foundational Models](https://mlsanity.com/paper/2602.12734/) (2026)
- [Social Neuro AI: Social Interaction as the "dark matter" of AI](https://mlsanity.com/paper/2112.15459/) (2021)
- [Implicit Affordance Acquisition via Causal Action-Effect Modeling in the Video Domain](https://mlsanity.com/paper/2312.11345/) (2023)
- [Affordance Perception by a Knowledge-Guided Vision-Language Model with Efficient Error Correction](https://mlsanity.com/paper/2407.13368/) (2024)
- [arg-VU: Affordance Reasoning with Physics-Aware 3D Geometry for Visual Understanding in Robotic Surgery](https://mlsanity.com/paper/2603.26814/) (2026)

## Abstract (arXiv)

Affordances describe the possibilities for an agent to perform actions with an object. While the significance of the affordance concept has been previously studied from varied perspectives, such as psychology and cognitive science, these approaches are not always sufficient to enable direct transfer, in the sense of implementations, to artificial intelligence (AI)-based systems and robotics. However, many efforts have been made to pragmatically employ the concept of affordances, as it represents great potential for AI agents to effectively bridge perception to action. In this survey, we review and find common ground amongst different strategies that use the concept of affordances within robotic tasks, and build on these methods to provide guidance for including affordances as a mechanism to improve autonomy. To this end, we outline common design choices for building representations of affordance relations, and their implications on the generalisation capabilities of an agent when facing previously unseen scenarios. Finally, we identify and discuss a range of interesting research directions involving affordances that have the potential to improve the capabilities of an AI agent.