mlsanityData to 7 Oct 2026

Grounded Human-Object Interaction Hotspots from Video

arXiv 1812.04558 ↗First posted 11 Dec 2018cs.CVcited by 128

Not on the map. Why?

Abstract (arXiv)

Learning how to interact with objects is an important step towards embodied visual intelligence, but existing techniques suffer from heavy supervision or sensing requirements. We propose an approach to learn human-object interaction "hotspots" directly from video. Rather than treat affordances as a manually supervised semantic segmentation task, our approach learns about interactions by watching videos of real human behavior and anticipating afforded actions. Given a novel image or video, our model infers a spatial hotspot map indicating how an object would be manipulated in a potential interaction-- even if the object is currently at rest. Through results with both first and third person video, we show the value of grounding affordances in real human-object interactions. Not only are our weakly supervised hotspots competitive with strongly supervised affordance methods, but they can also anticipate object interaction for novel object categories.

Key facts. Cited by 128 papers in the map (data through 7 Oct 2026). First posted on arXiv 11 Dec 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 14; 2022: 20; 2023: 24; 2024: 32; 2025: 24; 2026: 14.

128 papers.

Sort:

By area:

  1. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  2. 2023Affordances from Human Videos as a Versatile Representation for Roboticscited by 221
  3. 2021Where2Act: From Pixels to Actions for Articulated 3D Objectscited by 179
  4. 2023Structured World Models from Human Videoscited by 144
  5. 2024Track2Act: Predicting Point Tracks from Internet Videos enables Generalizable Robot Manipulationcited by 116
  6. 2022EgoTaskQA: Understanding Human Tasks in Egocentric Videoscited by 112
  7. 2022Joint Hand Motion and Interaction Hotspots Prediction from Egocentric Videoscited by 110
  8. 2023Affordance Diffusion: Synthesizing Hand-Object Interactionscited by 110
  9. 2023An Outlook into the Future of Egocentric Visioncited by 84
  10. 2022DexVIP: Learning Dexterous Grasping with Human Hand Pose Priors from Videocited by 83
  11. 2022Learning Affordance Grounding from Exocentric Imagescited by 82
  12. 2024Move as You Say, Interact as You Can: Language-guided Human Motion Generation with Scene Affordancecited by 79
38 more
  1. 2021VAT-Mart: Learning Visual Action Trajectory Proposals for Manipulating 3D ARTiculated Objectscited by 78
  2. 2022Fine-Grained Egocentric Hand-Object Segmentation: Dataset, Model, and Applicationscited by 78
  3. 2021Augmenting Reinforcement Learning with Behavior Primitives for Diverse Manipulation Taskscited by 75
  4. 2022Scene Graph Generation: A Comprehensive Surveycited by 73
  5. 2024Robo-ABC: Affordance Generalization Beyond Categories via Semantic Correspondence for Robot Manipulationcited by 69
  6. 2021UMPNet: Universal Manipulation Policy Network for Articulated Objectscited by 64
  7. 2023LOCATE: Localize and Transfer Object Parts for Weakly Supervised Affordance Groundingcited by 64
  8. 2023Grounding 3D Object Affordance from 2D Interactions in Imagescited by 62
  9. 2023Grounded Question-Answering in Long Egocentric Videoscited by 62
  10. 2024AffordanceLLM: Grounding Affordance from Vision Language Modelscited by 55
  11. 2022End-to-End Affordance Learning for Robotic Manipulationcited by 54
  12. 2025Open-Vocabulary Functional 3D Scene Graphs for Real-World Indoor Spacescited by 53
  13. 2021Human Hands as Probes for Interactive Object Understandingcited by 51
  14. 2023Towards Generalizable Zero-Shot Manipulation via Translating Human Interaction Planscited by 51
  15. 2023Zero-Shot Robot Manipulation from Passive Human Videoscited by 49
  16. 2023LEMON: Learning 3D Human-Object Interaction Relation from 2D Imagescited by 49
  17. 2021Forecasting Action through Contact Representations from First Person Videocited by 46
  18. 2021O2O-Afford: Annotation-Free Large-Scale Object-Object Affordance Learningcited by 46
  19. 2025VidBot: Learning Generalizable 3D Actions from In-the-Wild 2D Human Videos for Zero-Shot Robotic Manipulationcited by 46
  20. 2023One-Shot Open Affordance Learning with Foundation Modelscited by 45
  21. 2024G-HOP: Generative Hand-Object Prior for Interaction Reconstruction and Grasp Synthesiscited by 43
  22. 2023Detecting Human-Object Contact in Imagescited by 41
  23. 2021One-Shot Object Affordance Detection in the Wildcited by 40
  24. 2023Ditto in the House: Building Articulation Models of Indoor Scenes through Interactive Perceptioncited by 40
  25. 2023Egocentric Audio-Visual Object Localizationcited by 40
  26. 2024HRP: Human Affordances for Robotic Pre-Trainingcited by 40
  27. 2024Splat-MOVER: Multi-Stage, Open-Vocabulary Robotic Manipulation via Editable Gaussian Splattingcited by 37
  28. 2021Predicting the Future from First Person (Egocentric) Vision: A Surveycited by 36
  29. 2022MECCANO: A Multimodal Egocentric Dataset for Humans Behavior Understanding in the Industrial-like Domaincited by 36
  30. 2022PartAfford: Part-level Affordance Discovery from 3D Objectscited by 35
  31. 2023Affordance Grounding from Demonstration Video to Target Imagecited by 34
  32. 2024Learning Precise Affordances from Egocentric Videos for Robotic Manipulationcited by 34
  33. 2024A Survey of Embodied Learning for Object-Centric Robotic Manipulationcited by 34
  34. 2023Understanding 3D Object Interaction from a Single Imagecited by 33
  35. 2024GigaHands: A Massive Annotated Dataset of Bimanual Hand Activitiescited by 32
  36. 2022Egocentric Prediction of Action Target in 3Dcited by 31
  37. 20253D-AffordanceLLM: Harnessing Large Language Models for Open-Vocabulary Affordance Detection in 3D Worldscited by 29
  38. 2022Affordance Learning from Play for Sample-Efficient Policy Learningcited by 27

50 most cited of 128 All 128 (JSON)