# Learning to Act Properly: Predicting and Explaining Affordances from Images

arXiv: https://arxiv.org/abs/1712.07576 · HTML: https://mlsanity.com/paper/1712.07576/

## Key facts

- Cited by 74 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Dec 2017.

## Built on by (most cited first, 30 of 74)

- [From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI](https://mlsanity.com/paper/2201.08164/) (2022)
- [3D AffordanceNet: A Benchmark for Visual Object Affordance Understanding](https://mlsanity.com/paper/2103.16397/) (2021)
- [Learning Affordance Grounding from Exocentric Images](https://mlsanity.com/paper/2203.09905/) (2022)
- [Diffusion-Guided Reconstruction of Everyday Hand-Object Interaction Clips](https://mlsanity.com/paper/2309.05663/) (2023)
- [LOCATE: Localize and Transfer Object Parts for Weakly Supervised Affordance Grounding](https://mlsanity.com/paper/2303.09665/) (2023)
- [Grounding 3D Object Affordance from 2D Interactions in Images](https://mlsanity.com/paper/2303.10437/) (2023)
- [Putting People in Their Place: Affordance-Aware Human Insertion into Scenes](https://mlsanity.com/paper/2304.14406/) (2023)
- [VidBot: Learning Generalizable 3D Actions from In-the-Wild 2D Human Videos for Zero-Shot Robotic Manipulation](https://mlsanity.com/paper/2503.07135/) (2025)
- [One-Shot Open Affordance Learning with Foundation Models](https://mlsanity.com/paper/2311.17776/) (2023)
- [Open-Vocabulary Affordance Detection in 3D Point Clouds](https://mlsanity.com/paper/2303.02401/) (2023)
- [One-Shot Object Affordance Detection in the Wild](https://mlsanity.com/paper/2108.03658/) (2021)
- [Learning Precise Affordances from Egocentric Videos for Robotic Manipulation](https://mlsanity.com/paper/2408.10123/) (2024)
- [One-Shot Affordance Detection](https://mlsanity.com/paper/2106.14747/) (2021)
- [LODE: Locally Conditioned Eikonal Implicit Scene Completion from Sparse LiDAR](https://mlsanity.com/paper/2302.14052/) (2023)
- [Doubly Right Object Recognition: A Why Prompt for Visual Rationales](https://mlsanity.com/paper/2212.06202/) (2022)
- [Position: Measure Dataset Diversity, Don't Just Claim It](https://mlsanity.com/paper/2407.08188/) (2024)
- [TOIST: Task Oriented Instance Segmentation Transformer with Noun-Pronoun Distillation](https://mlsanity.com/paper/2210.10775/) (2022)
- [Grounded Affordance from Exocentric View](https://mlsanity.com/paper/2208.13196/) (2022)
- [Learning 2D Invariant Affordance Knowledge for 3D Affordance Grounding](https://mlsanity.com/paper/2408.13024/) (2024)
- [Scene Synthesis from Human Motion](https://mlsanity.com/paper/2301.01424/) (2023)
- [Multi-label affordance mapping from egocentric vision](https://mlsanity.com/paper/2309.02120/) (2023)
- [GEAL: Generalizable 3D Affordance Learning with Cross-Modal Consistency](https://mlsanity.com/paper/2412.09511/) (2024)
- [Scene-Driven Multimodal Knowledge Graph Construction for Embodied AI](https://mlsanity.com/paper/2311.03783/) (2023)
- [Intent3D: 3D Object Detection in RGB-D Scans Based on Human Intention](https://mlsanity.com/paper/2405.18295/) (2024)
- [RIO: A Benchmark for Reasoning Intention-Oriented Objects in Open Environments](https://mlsanity.com/paper/2310.17290/) (2023)
- [Hallucinating Pose-Compatible Scenes](https://mlsanity.com/paper/2112.06909/) (2021)
- [WorldAfford: Affordance Grounding based on Natural Language Instructions](https://mlsanity.com/paper/2405.12461/) (2024)
- [GLOVER++: Unleashing the Potential of Affordance Learning from Human Behaviors for Robotic Manipulation](https://mlsanity.com/paper/2505.11865/) (2025)
- [Beyond Object Recognition: A New Benchmark towards Object Concept Learning](https://mlsanity.com/paper/2212.02710/) (2022)
- [Text2Place: Affordance-aware Text Guided Human Placement](https://mlsanity.com/paper/2407.15446/) (2024)

## Abstract (arXiv)

We address the problem of affordance reasoning in diverse scenes that appear in the real world. Affordances relate the agent's actions to their effects when taken on the surrounding objects. In our work, we take the egocentric view of the scene, and aim to reason about action-object affordances that respect both the physical world as well as the social norms imposed by the society. We also aim to teach artificial agents why some actions should not be taken in certain situations, and what would likely happen if these actions would be taken. We collect a new dataset that builds upon ADE20k, referred to as ADE-Affordance, which contains annotations enabling such rich visual reasoning. We propose a model that exploits Graph Neural Networks to propagate contextual information from the scene in order to perform detailed affordance reasoning about each object. Our model is showcased through various ablation studies, pointing to successes and challenges in this complex task.