# Contrastively Learning Visual Attention as Affordance Cues from Demonstrations for Robotic Grasping

arXiv: https://arxiv.org/abs/2104.00878 · HTML: https://mlsanity.com/paper/2104.00878/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 39 references that are in the map.
- First posted on arXiv 2 Apr 2021.

## Builds on (most cited first, 10 of 10)

- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [See More, Know More: Unsupervised Video Object Segmentation with Co-Attention Siamese Networks](https://mlsanity.com/paper/2001.06810/) (2020)
- [Robotic Pick-and-Place of Novel Objects in Clutter with Multi-Affordance Grasping and Cross-Domain Image Matching](https://mlsanity.com/paper/1710.01330/) (2017)
- [Occlusion Robust Face Recognition Based on Mask Learning with PairwiseDifferential Siamese Network](https://mlsanity.com/paper/1908.06290/) (2019)
- [Simultaneous Policy Learning and Latent State Inference for Imitating Driver Behavior](https://mlsanity.com/paper/1704.05566/) (2017)
- [Pay attention! - Robustifying a Deep Visuomotor Policy through Task-Focused Attention](https://mlsanity.com/paper/1809.10093/) (2018)
- [A Survey of Embedding Space Alignment Methods for Language and Knowledge Graphs](https://mlsanity.com/paper/2010.13688/) (2020)
- [Object cosegmentation using deep Siamese network](https://mlsanity.com/paper/1803.02555/) (2018)
- [Generalization Guarantees for Imitation Learning](https://mlsanity.com/paper/2008.01913/) (2020)
- [Large Scale Novel Object Discovery in 3D](https://mlsanity.com/paper/1701.07046/) (2017)

## Built on by (most cited first, 7 of 7)

- [AffordDP: Generalizable Diffusion Policy with Transferable Affordance](https://mlsanity.com/paper/2412.03142/) (2024)
- [Learning Visual Affordance Grounding from Demonstration Videos](https://mlsanity.com/paper/2108.05675/) (2021)
- [UAD: Unsupervised Affordance Distillation for Generalization in Robotic Manipulation](https://mlsanity.com/paper/2506.09284/) (2025)
- [Text-driven Affordance Learning from Egocentric Vision](https://mlsanity.com/paper/2404.02523/) (2024)
- [Robot Manipulation in Salient Vision through Referring Image Segmentation and Geometric Constraints](https://mlsanity.com/paper/2409.11518/) (2024)
- [Exploiting Unlabeled Data for Feedback Efficient Human Preference based Reinforcement Learning](https://mlsanity.com/paper/2302.08738/) (2023)
- [O3Afford: One-Shot 3D Object-to-Object Affordance Grounding for Generalizable Robotic Manipulation](https://mlsanity.com/paper/2509.06233/) (2025)

## Abstract (arXiv)

Conventional works that learn grasping affordance from demonstrations need to explicitly predict grasping configurations, such as gripper approaching angles or grasping preshapes. Classic motion planners could then sample trajectories by using such predicted configurations. In this work, our goal is instead to fill the gap between affordance discovery and affordance-based policy learning by integrating the two objectives in an end-to-end imitation learning framework based on deep neural networks. From a psychological perspective, there is a close association between attention and affordance. Therefore, with an end-to-end neural network, we propose to learn affordance cues as visual attention that serves as a useful indicating signal of how a demonstrator accomplishes tasks, instead of explicitly modeling affordances. To achieve this, we propose a contrastive learning framework that consists of a Siamese encoder and a trajectory decoder. We further introduce a coupled triplet loss to encourage the discovered affordance cues to be more affordance-relevant. Our experimental results demonstrate that our model with the coupled triplet loss achieves the highest grasping success rate in a simulated robot environment. Our project website can be accessed at https://sites.google.com/asu.edu/affordance-aware-imitation/project.