Visual-Geometric Collaborative Guidance for Affordance Learning
Not on the map. Why?
Abstract (arXiv)
Perceiving potential ``action possibilities'' (ie, affordance) regions of images and learning interactive functionalities of objects from human demonstration is a challenging task due to the diversity of human-object interactions. Prevailing affordance learning algorithms often adopt the label assignment paradigm and presume that there is a unique relationship between functional region and affordance label, yielding poor performance when adapting to unseen environments with large appearance variations. In this paper, we propose to leverage interactive affinity for affordance learning, ie extracting interactive affinity from human-object interaction and transferring it to non-interactive objects. Interactive affinity, which represents the contacts between different parts of the human body and local regions of the target object, can provide inherent cues of interconnectivity between humans and objects, thereby reducing the ambiguity of the perceived action possibilities. To this end, we propose a visual-geometric collaborative guided affordance learning network that incorporates visual and geometric cues to excavate interactive affinity from human-object interactions jointly. Besides, a contact-driven affordance learning (CAL) dataset is constructed by collecting and labeling over 55,047 images. Experimental results demonstrate that our method outperforms the representative models regarding objective metrics and visual quality. Project: hrefhttps://github.com/lhc1224/VCR-Netgithub.com/lhc1224/VCR-Net.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 52 of its 81 references that are in the map. First posted on arXiv 15 Oct 2024.
Builds on
52 of 81 references.
References · 52
- 2017Attention Is All You Needcited by 60,361
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2014Microsoft COCO: Common Objects in Contextcited by 17,608
- 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,223
- 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,940
- 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
- 2016Pyramid Scene Parsing Networkcited by 2,479
- 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
- 2018Datasheets for Datasetscited by 1,386
- 2019Deep Equilibrium Modelscited by 615
- 2022ViTPose: Simple Vision Transformer Baselines for Human Pose Estimationcited by 456
38 more
- 2022BEHAVE: Dataset and Method for Tracking Human Object Interactionscited by 249
- 2017Learning to Detect Human-Object Interactionscited by 240
- 2015DeepDriving: Learning Affordance for Direct Perception in Autonomous Drivingcited by 231
- 2017Detecting and Recognizing Human-Object Interactionscited by 199
- 2021Hypercorrelation Squeeze for Few-Shot Segmentationcited by 193
- 2016What do different evaluation metrics tell us about saliency models?cited by 185
- 2021HOTR: End-to-End Human-Object Interaction Detection with Transformerscited by 183
- 2023ManipLLM: Embodied Multimodal Large Language Model for Object-Centric Robotic Manipulationcited by 162
- 2023SMPLer-X: Scaling Up Expressive Human Pose and Shape Estimationcited by 155
- 2021HRFormer: High-Resolution Transformer for Dense Predictioncited by 149
- 2022GAPartNet: Cross-Category Domain-Generalizable Object Perception and Manipulation via Generalizable and Actionable Partscited by 139
- 2018Grounded Human-Object Interaction Hotspots from Videocited by 129
- 2017AffordanceNet: An End-to-End Deep Learning Approach for Object Affordance Detectioncited by 128
- 2020CPF: Learning a Contact Potential Field to Model the Hand-Object Interactioncited by 128
- 20213D AffordanceNet: A Benchmark for Visual Object Affordance Understandingcited by 100
- 2022Learning Affordance Grounding from Exocentric Imagescited by 82
- 2017Learning to Act Properly: Predicting and Explaining Affordances from Imagescited by 74
- 2020Learning Affordance Landscapes for Interaction Exploration in 3D Environmentscited by 65
- 2023LOCATE: Localize and Transfer Object Parts for Weakly Supervised Affordance Groundingcited by 64
- 2023Grounding 3D Object Affordance from 2D Interactions in Imagescited by 62
- 2021AdaAfford: Learning to Adapt Manipulation Affordance for 3D Articulated Objects via Few-shot Interactionscited by 57
- 2018Binge Watching: Scaling Affordance Learning from Sitcomscited by 41
- 2023Detecting Human-Object Contact in Imagescited by 41
- 2021One-Shot Object Affordance Detection in the Wildcited by 40
- 2023CoTDet: Affordance Knowledge Prompting for Task Driven Object Detectioncited by 36
- 2023Where2Explore: Few-shot Affordance Learning for Unseen Novel Categories of Articulated Objectscited by 35
- 2023Affordance Grounding from Demonstration Video to Target Imagecited by 34
- 2021One-Shot Affordance Detectioncited by 32
- 2022HULC: 3D Human Motion Capture with Pose Manifold Sampling and Dense Contact Guidancecited by 29
- 2022Contact-aware Human Motion Forecastingcited by 27
- 2017Predicting Human Activities Using Stochastic Grammarcited by 22
- 2022Grounded Affordance from Exocentric Viewcited by 22
- 2018Visual Affordance and Function Understanding: A Surveycited by 20
- 2017Adaptive Binarization for Weakly Supervised Affordance Segmentationcited by 16
- 2022Learning Agent-Aware Affordances for Closed-Loop Interaction with Articulated Objectscited by 16
- 2023RIO: A Benchmark for Reasoning Intention-Oriented Objects in Open Environmentscited by 15
- 2024PEAR: Phrase-Based Hand-Object Interaction Anticipationcited by 10
- 2024Bidirectional Progressive Transformer for Interaction Intention Anticipationcited by 6
50 most cited of 52 All 52 (JSON)
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.