mlsanityData to 7 Oct 2026

Affordance Grounding from Demonstration Video to Target Image

arXiv 2303.14644 ↗First posted 26 Mar 2023cs.CVcited by 34

Not on the map. Why?

Abstract (arXiv)

Humans excel at learning from expert demonstrations and solving their own problems. To equip intelligent robots and assistants, such as AR glasses, with this ability, it is essential to ground human hand interactions (i.e., affordances) from demonstration videos and apply them to a target image like a user's AR glass view. The video-to-image affordance grounding task is challenging due to (1) the need to predict fine-grained affordances, and (2) the limited training data, which inadequately covers video-image discrepancies and negatively impacts grounding. To tackle them, we propose Affordance Transformer (Afformer), which has a fine-grained transformer-based decoder that gradually refines affordance grounding. Moreover, we introduce Mask Affordance Hand (MaskAHand), a self-supervised pre-training technique for synthesizing video-image data and simulating context changes, enhancing affordance grounding across video-image discrepancies. Afformer with MaskAHand pre-training achieves state-of-the-art performance on multiple benchmarks, including a substantial 37% improvement on the OPRA dataset. Code is made available at https://github.com/showlab/afformer.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). Builds on 48 of its 62 references that are in the map. First posted on arXiv 26 Mar 2023.

Builds on

48 of 62 references.

References · 48

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  6. 2020End-to-End Object Detection with Transformerscited by 7,384
  7. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  8. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  9. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  10. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  11. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
  12. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
36 more
  1. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  2. 2017The Kinetics Human Action Video Datasetcited by 1,894
  3. 2018SlowFast Networks for Video Recognitioncited by 1,862
  4. 2017SuperPoint: Self-Supervised Interest Point Detection and Descriptioncited by 1,313
  5. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,148
  6. 2022Exploring Plain Vision Transformer Backbones for Object Detectioncited by 723
  7. 2021Masked Feature Prediction for Self-Supervised Visual Pre-Trainingcited by 573
  8. 2021MViTv2: Improved Multiscale Vision Transformers for Classification and Detectioncited by 516
  9. 2022Masked Autoencoders As Spatiotemporal Learnerscited by 428
  10. 2020Understanding Human Hands in Contact at Internet Scalecited by 281
  11. 2022Egocentric Video-Language Pretrainingcited by 231
  12. 2019Cross Attention Network for Few-shot Classificationcited by 218
  13. 2016What do different evaluation metrics tell us about saliency models?cited by 184
  14. 2018Learning Human-Object Interactions by Graph Parsing Neural Networkscited by 176
  15. 2018Grounded Human-Object Interaction Hotspots from Videocited by 128
  16. 2017AffordanceNet: An End-to-End Deep Learning Approach for Object Affordance Detectioncited by 126
  17. 2022Joint Hand Motion and Interaction Hotspots Prediction from Egocentric Videoscited by 110
  18. 2018Actor and Observer: Joint Modeling of First and Third-Person Videoscited by 102
  19. 20213D AffordanceNet: A Benchmark for Visual Object Affordance Understandingcited by 100
  20. 2022DexVIP: Learning Dexterous Grasping with Human Hand Pose Priors from Videocited by 83
  21. 2022Learning Affordance Grounding from Exocentric Imagescited by 82
  22. 2020Learning Dexterous Grasping with Object-Centric Visual Affordancescited by 80
  23. 2018Predicting Gaze in Egocentric Video by Learning Task-dependent Attention Transitioncited by 78
  24. 2017SalGAN: Visual Saliency Prediction with Generative Adversarial Networkscited by 63
  25. 2021Object-aware Video-language Pre-training for Retrievalcited by 57
  26. 2019Next-Active-Object prediction from Egocentric Videoscited by 55
  27. 2016A Deep Multi-Level Network for Saliency Predictioncited by 54
  28. 2016DeepGaze II: Reading fixations from deep features trained on object recognitioncited by 54
  29. 2021Human Hands as Probes for Interactive Object Understandingcited by 51
  30. 2021One-Shot Object Affordance Detection in the Wildcited by 40
  31. 2022AssistQ: Affordance-centric Question-driven Task Completion for Egocentric Assistantcited by 32
  32. 2021Learning Visual Affordance Grounding from Demonstration Videoscited by 25
  33. 2021MorphMLP: An Efficient MLP-Like Backbone for Spatial-Temporal Representation Learningcited by 21
  34. 2019Deep Saliency Models : The Quest For The Loss Functioncited by 8
  35. 2021AssistSR: Task-oriented Video Segment Retrieval for Personal AI Assistantcited by 8
  36. 2021CAT: Cross-Attention Transformer for One-Shot Object Detectioncited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 10; 2025: 12; 2026: 9.

34 papers.

Sort:

By area:

  1. 2023AssistGPT: A General Multi-modal Assistant that can Plan, Execute, Inspect, and Learncited by 97
  2. 2024DoraemonGPT: Toward Understanding Dynamic Scenes with Large Language Models (Exemplified as A Video Agent)cited by 56
  3. 2024AffordanceLLM: Grounding Affordance from Vision Language Modelscited by 55
  4. 2023One-Shot Open Affordance Learning with Foundation Modelscited by 45
  5. 2024Learning Manipulation by Predicting Interactioncited by 36
  6. 20253D-AffordanceLLM: Harnessing Large Language Models for Open-Vocabulary Affordance Detection in 3D Worldscited by 29
  7. 2026PALM: Progress-Aware Policy Learning via Affordance Reasoning for Long-Horizon Robotic Manipulationcited by 20
  8. 2024GEAL: Generalizable 3D Affordance Learning with Cross-Modal Consistencycited by 17
  9. 2024WorldAfford: Affordance Grounding based on Natural Language Instructionscited by 14
  10. 2024INTRA: Interaction Relationship-aware Weakly Supervised Affordance Groundingcited by 12
  11. 2025RAGNet: Large-scale Reasoning-based Affordance Segmentation Benchmark towards General Graspingcited by 11
  12. 2024Text-driven Affordance Learning from Egocentric Visioncited by 9
22 more
  1. 2025Weakly-Supervised Affordance Grounding Guided by Part-Level Semantic Priorscited by 9
  2. 2025EgoWorld: Translating Exocentric View to Egocentric View using Rich Exocentric Observationscited by 8
  3. 2024Egocentric and Exocentric Methods: A Short Surveycited by 6
  4. 2025AnchorDP3: 3D Affordance Guided Sparse Diffusion Policy for Robotic Manipulationcited by 4
  5. 2025Closed-Loop Transfer for Weakly-supervised Affordance Groundingcited by 4
  6. 2023First Place Solution to the CVPR'2023 AQTC Challenge: A Function-Interaction Centric Approach with Spatiotemporal Visual-Language Alignmentcited by 3
  7. 2025Interpretable Affordance Detection on 3D Point Clouds with Probabilistic Prototypescited by 3
  8. 2024Visual-Geometric Collaborative Guidance for Affordance Learningcited by 2
  9. 2026VideoAfford: Grounding 3D Affordance from Human-Object-Interaction Videos via Multimodal Large Language Modelcited by 2
  10. 2025H2OFlow: Grounding Human-Object Affordances with 3D Generative Models and Dense Diffused Flowscited by 1
  11. 2026VAGNet: Grounding 3D Affordance from Human-Object Interactions in Videoscited by 1
  12. 2026CompassAD: Intent-Driven 3D Affordance Grounding in Functionally Competing Objectscited by 1
  13. 2024Leverage Task Context for Object Affordance Rankingcited by 0
  14. 2025Capturing Fine-Grained Alignments Improves 3D Affordance Detectioncited by 0
  15. 2025Populate-A-Scene: Affordance-Aware Human Video Generationcited by 0
  16. 2025TReF-6: Inferring Task-Relevant Frames from a Single Demonstration for One-Shot Skill Generalizationcited by 0
  17. 2025Learning Visual Affordance from Audiocited by 0
  18. 2026Probing and Bridging Geometry-Interaction Cues for Affordance Reasoning in Vision Foundation Modelscited by 0
  19. 2026AffordMatcher: Affordance Learning in 3D Scenes from Visual Signifierscited by 0
  20. 2026What Objects Enable, Not What They Are: Functional Latent Spaces for Affordance Reasoningcited by 0
  21. 2026HAT-4D: Lifting Monocular Video for 4D Multi-Object Interactions via Human-Agent Collaborationcited by 0
  22. 2026Zero-shot 2D Grounding with Novel Affordance Typescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Affordance2Action: Task-Conditioned Scene-level Affordance Grounding for Real-Time Manipulation ↗
  2. 2025UAD: Unsupervised Affordance Distillation for Generalization in Robotic Manipulation
  3. 2026SpatialAfford: Teaching Compact VLMs Where to Look and Where to Ground for Affordance
  4. 20252HandedAfforder: Learning Precise Actionable Bimanual Affordances from Human Videos
  5. 2025Grounding 3D Object Affordance with Language Instructions, Visual Observations and Interactions
  6. 2026RAAP: Retrieval-Augmented Affordance Prediction with Cross-Image Action Alignment
  7. 2026ThinkAfford: Affordance-Centric Reasoning for Fine-Grained 3D Grounding in Cluttered Scenes
  8. 2026Token-Based Affordance Grounding with Large Vision-Language Models