mlsanityData to 7 Oct 2026

Hand-Object Interaction Reasoning

arXiv 2201.04906 ↗First posted 13 Jan 2022cs.CVcited by 7

Not on the map. Why?

Abstract (arXiv)

This paper proposes an interaction reasoning network for modelling spatio-temporal relationships between hands and objects in video. The proposed interaction unit utilises a Transformer module to reason about each acting hand, and its spatio-temporal relation to the other hand as well as objects being interacted with. We show that modelling two-handed interactions are critical for action recognition in egocentric video, and demonstrate that by using positionally-encoded trajectories, the network can better recognise observed interactions. We evaluate our proposal on EPIC-KITCHENS and Something-Else datasets, with an ablation study.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 38 of its 45 references that are in the map. First posted on arXiv 13 Jan 2022.

Builds on

38 of 45 references.

References · 38

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  3. 2017Mask R-CNNcited by 7,259
  4. 2020Longformer: The Long-Document Transformercited by 3,362
  5. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  6. 2017Non-local Neural Networkscited by 2,290
  7. 2017The Kinetics Human Action Video Datasetcited by 1,894
  8. 2018SlowFast Networks for Video Recognitioncited by 1,862
  9. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,609
  10. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,441
  11. 2017The "something something" video database for learning and evaluating visual common sensecited by 1,092
  12. 2014Two-Stream Convolutional Networks for Action Recognition in Videoscited by 1,067
26 more
  1. 2016Temporal Segment Networks: Towards Good Practices for Deep Action Recognitioncited by 899
  2. 2018TSM: Temporal Shift Module for Efficient Video Understandingcited by 852
  3. 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 789
  4. 2020X3D: Expanding Architectures for Efficient Video Recognitioncited by 635
  5. 2014Long-term Recurrent Convolutional Networks for Visual Recognition and Descriptioncited by 483
  6. 2017AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actionscited by 449
  7. 2016Convolutional Two-Stream Network Fusion for Video Action Recognitioncited by 384
  8. 2017Temporal Relational Reasoning in Videoscited by 324
  9. 2018Video Action Transformer Networkcited by 316
  10. 2020Understanding Human Hands in Contact at Internet Scalecited by 281
  11. 2021Video Transformer Networkcited by 271
  12. 2018Long-Term Feature Banks for Detailed Video Understandingcited by 239
  13. 2017Learning to Detect Human-Object Interactionscited by 238
  14. 2015Beyond Short Snippets: Deep Networks for Video Classificationcited by 233
  15. 2018Videos as Space-Time Region Graphscited by 221
  16. 2017Detecting and Recognizing Human-Object Interactionscited by 199
  17. 2020How Much Position Information Do Convolutional Neural Networks Encode?cited by 171
  18. 2018iCAN: Instance-Centric Attention Network for Human-Object Interaction Detectioncited by 126
  19. 2020DRG: Dual Relation Graph for Human-Object Interaction Detectioncited by 117
  20. 2019Something-Else: Compositional Action Recognition with Spatial-Temporal Interaction Networkscited by 112
  21. 2020Actor-Context-Actor Relation Network for Spatio-Temporal Action Localizationcited by 91
  22. 2019Pose-aware Multi-level Feature Network for Human Object Interaction Detectioncited by 88
  23. 2018Actor-Centric Relation Networkcited by 87
  24. 2018Object Level Visual Reasoning in Videoscited by 67
  25. 2019Collaborative Spatio-temporal Feature Learning for Video Action Recognitioncited by 24
  26. 2021Revisiting spatio-temporal layouts for compositional action recognitioncited by 19

Built on by

Cited by year

Citing papers per year: 2023: 4; 2024: 2; 2025: 1.

7 papers.

  1. 2023Learning Fine-grained View-Invariant Representations from Unpaired Ego-Exo Videos via Temporal Alignmentcited by 46
  2. 2023Opening the Vocabulary of Egocentric Actionscited by 23
  3. 2025Predicting 4D Hand Trajectory from Monocular Videoscited by 19
  4. 2024On the Utility of 3D Hand Poses for Action Recognitioncited by 17
  5. 2023Chop & Learn: Recognizing and Generating Object-State Compositionscited by 13
  6. 2024GPT4Ego: Unleashing the Potential of Pre-trained Models for Zero-Shot Egocentric Action Recognitioncited by 8
  7. 2023Are Labels Needed for Incremental Instance Learning?cited by 2