# Building an Affordances Map with Interactive Perception

arXiv: https://arxiv.org/abs/1903.04413 · HTML: https://mlsanity.com/paper/1903.04413/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Mar 2019.

## Built on by (most cited first, 6 of 6)

- [VAT-Mart: Learning Visual Action Trajectory Proposals for Manipulating 3D ARTiculated Objects](https://mlsanity.com/paper/2106.14440/) (2021)
- [Learning Foresightful Dense Visual Affordance for Deformable Object Manipulation](https://mlsanity.com/paper/2303.11057/) (2023)
- [From Screens to Scenes: A Survey of Embodied AI in Healthcare](https://mlsanity.com/paper/2501.07468/) (2025)
- [Real-time Mapping of Physical Scene Properties with an Autonomous Robot Experimenter](https://mlsanity.com/paper/2210.17325/) (2022)
- [SCOD: Active Object Detection for Embodied Agents using Sensory Commutativity of Action Sequences](https://mlsanity.com/paper/2107.02069/) (2021)
- [Object-Centric Action-Enhanced Representations for Robot Visuo-Motor Policy Learning](https://mlsanity.com/paper/2505.20962/) (2025)

## Abstract (arXiv)

Robots need to understand their environment to perform their task. If it is possible to pre-program a visual scene analysis process in closed environments, robots operating in an open environment would benefit from the ability to learn it through their interaction with their environment. This ability furthermore opens the way to the acquisition of affordances maps in which the action capabilities of the robot structure its visual scene understanding. We propose an approach to build such affordances maps by relying on an interactive perception approach and an online classification. In the proposed formalization of affordances, actions and effects are related to visual features, not objects, and they can be combined. We have tested the approach on three action primitives and on a real PR2 robot.