mlsanityData to 14 Oct 2026

HOI4ABOT: Human-Object Interaction Anticipation for Human Intention Reading Collaborative roBOTs

arXiv 2309.16524 ↗First posted 28 Sep 2023cs.CV cs.ROcited by 12

Not on the map. Why?

Abstract (arXiv)

Robots are becoming increasingly integrated into our lives, assisting us in various tasks. To ensure effective collaboration between humans and robots, it is essential that they understand our intentions and anticipate our actions. In this paper, we propose a Human-Object Interaction (HOI) anticipation framework for collaborative robots. We propose an efficient and robust transformer-based model to detect and anticipate HOIs from videos. This enhanced anticipation empowers robots to proactively assist humans, resulting in more efficient and intuitive collaborations. Our model outperforms state-of-the-art results in HOI detection and anticipation in VidHOI dataset with an increase of 1.76% and 1.04% in mAP respectively while being 15.4 times faster. We showcase the effectiveness of our approach through experimental results in a real robot, demonstrating that the robot's ability to anticipate HOIs is key for better Human-Robot Interaction. More information can be found on our project webpage: https://evm7.github.io/HOI4ABOT_page/

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 28 of its 57 references that are in the map. First posted on arXiv 28 Sep 2023.

Builds on

28 of 57 references.

References · 28

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  5. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,200
  6. 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,245
  7. 2022ViTPose: Simple Vision Transformer Baselines for Human Pose Estimationcited by 456
  8. 2016Image Captioning with Semantic Attentioncited by 260
  9. 2019Action Genome: Actions as Composition of Spatio-temporal Scene Graphscited by 250
  10. 2015Visual Semantic Role Labelingcited by 190
  11. 2021QPIC: Query-Based Pairwise Human-Object Interaction Detection with Image-Wide Contextual Informationcited by 147
  12. 2022GEN-VLKT: Simplify Association and Enhance Interaction Understanding for HOI Detectioncited by 120
16 more
  1. 2020VSGNet: Spatial Attention Network for Detecting Human Object Interactions Using Graph Convolutionscited by 115
  2. 2017Behavior Trees in Robotics and AI: An Introductioncited by 107
  3. 2021Spatial-Temporal Transformer for Dynamic Scene Graph Generationcited by 96
  4. 2022TASKOGRAPHY: Evaluating robot task planning over large 3D scene graphscited by 59
  5. 2023ViPLO: Vision Transformer based Pose-Conditioned Self-Loop Graph for Human-Object Interaction Detectioncited by 41
  6. 2022Intention-Conditioned Long-Term Human Egocentric Action Forecastingcited by 35
  7. 2022Mining Cross-Person Cues for Body-Part Interactiveness Learning in HOI Detectioncited by 34
  8. 2020Object Handovers: a Review for Roboticscited by 28
  9. 2022Reasoning with Scene Graphs for Robot Planning under Partial Observabilitycited by 25
  10. 2023Robotic Vision for Human-Robot Interaction and Collaboration: A Survey and Systematic Reviewcited by 25
  11. 2016CoSTAR: Instructing Collaborative Robots with Behavior Trees and Visioncited by 20
  12. 2021ST-HOI: A Spatial-Temporal Baseline for Human-Object Interaction Detection in Videoscited by 20
  13. 2021Human-robot collaboration and machine learning: a systematic review of recent researchcited by 20
  14. 2022Proactive Robot Assistance via Spatio-Temporal Object Modelingcited by 18
  15. 2022Video-based Human-Object Interaction Detection from Tubelet Tokenscited by 13
  16. 2023Human-Object Interaction Prediction in Videos through Gaze Followingcited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 8; 2026: 3.

12 papers.

  1. 2025HUMOTO: A 4D Dataset of Mocap Human Object Interactionscited by 31
  2. 2025Perceiving and Acting in First-Person: A Dataset and Benchmark for Egocentric Human-Object-Human Interactionscited by 8
  3. 2025An Image-like Diffusion Method for Human-Object Interaction Detectioncited by 5
  4. 2025RHINO: Learning Real-Time Humanoid-Human-Object Interaction from Human Demonstrationscited by 4
  5. 2025SGC-Net: Stratified Granular Comparison Network for Open-Vocabulary HOI Detectioncited by 3
  6. 2024Robot Behavior Personalization from Sparse User Feedbackcited by 2
  7. 2025Foundation Model-Driven Framework for Human-Object Interaction Prediction with Segmentation Mask Integrationcited by 0
  8. 2025Incremental Human-Object Interaction Detection with Invariant Relation Representation Learningcited by 0
  9. 2025Generative Human-Object Interaction Detection via Differentiable Cognitive Steering of Multi-modal LLMscited by 0
  10. 2026Towards Unconstrained Human-Object Interactioncited by 0
  11. 2026SAGE: Synchronized Action-Gaze Recognition and Anticipation for Human Behavior Understandingcited by 0
  12. 2026RGB-D Video Generation for Improving Human-to-Robot Object Handover Predictioncited by 0