# Phrase-Based Affordance Detection via Cyclic Bilateral Interaction

arXiv: https://arxiv.org/abs/2202.12076 · HTML: https://mlsanity.com/paper/2202.12076/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 63 references that are in the map.
- First posted on arXiv 24 Feb 2022.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Generation and Comprehension of Unambiguous Object Descriptions](https://mlsanity.com/paper/1511.02283/) (2015)
- [MAttNet: Modular Attention Network for Referring Expression Comprehension](https://mlsanity.com/paper/1801.08186/) (2018)
- [Enhanced-alignment Measure for Binary Foreground Map Evaluation](https://mlsanity.com/paper/1805.10421/) (2018)
- [Segmentation from Natural Language Expressions](https://mlsanity.com/paper/1603.06180/) (2016)
- [Cascaded Partial Decoder for Fast and Accurate Salient Object Detection](https://mlsanity.com/paper/1904.08739/) (2019)
- [MUTAN: Multimodal Tucker Fusion for Visual Question Answering](https://mlsanity.com/paper/1705.06676/) (2017)
- [Grounded Human-Object Interaction Hotspots from Video](https://mlsanity.com/paper/1812.04558/) (2018)
- [Recurrent Multimodal Interaction for Referring Image Segmentation](https://mlsanity.com/paper/1703.07939/) (2017)
- [Cross-Modal Progressive Comprehension for Referring Segmentation](https://mlsanity.com/paper/2105.07175/) (2021)
- [Locate then Segment: A Strong Pipeline for Referring Image Segmentation](https://mlsanity.com/paper/2103.16284/) (2021)
- [Improving Referring Expression Grounding with Cross-modal Attention-guided Erasing](https://mlsanity.com/paper/1903.00839/) (2019)
- [Dynamic Multimodal Instance Segmentation guided by natural language queries](https://mlsanity.com/paper/1807.02257/) (2018)
- [A Joint Speaker-Listener-Reinforcer Model for Referring Expressions](https://mlsanity.com/paper/1612.09542/) (2016)
- [Social Scene Understanding: End-to-End Multi-Person Action Localization and Collective Activity Recognition](https://mlsanity.com/paper/1611.09078/) (2016)
- [One-Shot Object Affordance Detection in the Wild](https://mlsanity.com/paper/2108.03658/) (2021)
- [One-Shot Affordance Detection](https://mlsanity.com/paper/2106.14747/) (2021)
- [Referring Segmentation in Images and Videos with Cross-Modal Self-Attention Network](https://mlsanity.com/paper/2102.04762/) (2021)
- [Learning Visual Affordance Grounding from Demonstration Videos](https://mlsanity.com/paper/2108.05675/) (2021)
- [Predicting Human Activities Using Stochastic Grammar](https://mlsanity.com/paper/1708.00945/) (2017)
- [Background Activation Suppression for Weakly Supervised Object Localization](https://mlsanity.com/paper/2112.00580/) (2021)
- [Deep Affordance-grounded Sensorimotor Object Recognition](https://mlsanity.com/paper/1704.02787/) (2017)

## Built on by (most cited first, 19 of 19)

- [Learning Affordance Grounding from Exocentric Images](https://mlsanity.com/paper/2203.09905/) (2022)
- [Grounding 3D Object Affordance from 2D Interactions in Images](https://mlsanity.com/paper/2303.10437/) (2023)
- [3D-AffordanceLLM: Harnessing Large Language Models for Open-Vocabulary Affordance Detection in 3D Worlds](https://mlsanity.com/paper/2502.20041/) (2025)
- [GREAT: Geometry-Intention Collaborative Inference for Open-Vocabulary 3D Object Affordance Grounding](https://mlsanity.com/paper/2411.19626/) (2024)
- [EgoChoir: Capturing 3D Human-Object Interaction Regions from Egocentric Views](https://mlsanity.com/paper/2405.13659/) (2024)
- [Grounded Affordance from Exocentric View](https://mlsanity.com/paper/2208.13196/) (2022)
- [Learning 2D Invariant Affordance Knowledge for 3D Affordance Grounding](https://mlsanity.com/paper/2408.13024/) (2024)
- [GEAL: Generalizable 3D Affordance Learning with Cross-Modal Consistency](https://mlsanity.com/paper/2412.09511/) (2024)
- [Intent3D: 3D Object Detection in RGB-D Scans Based on Human Intention](https://mlsanity.com/paper/2405.18295/) (2024)
- [RIO: A Benchmark for Reasoning Intention-Oriented Objects in Open Environments](https://mlsanity.com/paper/2310.17290/) (2023)
- [WorldAfford: Affordance Grounding based on Natural Language Instructions](https://mlsanity.com/paper/2405.12461/) (2024)
- [Grounding 3D Scene Affordance From Egocentric Interactions](https://mlsanity.com/paper/2409.19650/) (2024)
- [Generating 6DoF Object Manipulation Trajectories from Action Description in Egocentric Vision](https://mlsanity.com/paper/2506.03605/) (2025)
- [Text-driven Affordance Learning from Egocentric Vision](https://mlsanity.com/paper/2404.02523/) (2024)
- [AnchorDP3: 3D Affordance Guided Sparse Diffusion Policy for Robotic Manipulation](https://mlsanity.com/paper/2506.19269/) (2025)
- [Afford-X: Generalizable and Slim Affordance Reasoning for Task-oriented Manipulation](https://mlsanity.com/paper/2503.03556/) (2025)
- [Implicit Affordance Acquisition via Causal Action-Effect Modeling in the Video Domain](https://mlsanity.com/paper/2312.11345/) (2023)
- [H2OFlow: Grounding Human-Object Affordances with 3D Generative Models and Dense Diffused Flows](https://mlsanity.com/paper/2510.21769/) (2025)
- [Leverage Task Context for Object Affordance Ranking](https://mlsanity.com/paper/2411.16082/) (2024)

## Abstract (arXiv)

Affordance detection, which refers to perceiving objects with potential action possibilities in images, is a challenging task since the possible affordance depends on the person's purpose in real-world application scenarios. The existing works mainly extract the inherent human-object dependencies from image/video to accommodate affordance properties that change dynamically. In this paper, we explore to perceive affordance from a vision-language perspective and consider the challenging phrase-based affordance detection problem,i.e., given a set of phrases describing the action purposes, all the object regions in a scene with the same affordance should be detected. To this end, we propose a cyclic bilateral consistency enhancement network (CBCE-Net) to align language and vision features progressively. Specifically, the presented CBCE-Net consists of a mutual guided vision-language module that updates the common features of vision and language in a progressive manner, and a cyclic interaction module (CIM) that facilitates the perception of possible interaction with objects in a cyclic manner. In addition, we extend the public Purpose-driven Affordance Dataset (PAD) by annotating affordance categories with short phrases. The contrastive experimental results demonstrate the superiority of our method over nine typical methods from four relevant fields in terms of both objective metrics and visual quality. The related code and dataset will be released at urlhttps://github.com/lulsheng/CBCE-Net.