Beyond Sequences: A Benchmark for Atomic Hand-Object Interaction Using a Static RNN Encoder
Not on the map. Why?
Abstract (arXiv)
Reliably predicting human intent in hand-object interactions is an open challenge for computer vision. Our research concentrates on a fundamental sub-problem: the fine-grained classification of atomic interaction states, namely 'approaching', 'grabbing', and 'holding'. To this end, we introduce a structured data engineering process that converts raw videos from the MANIAC dataset into 27,476 statistical-kinematic feature vectors. Each vector encapsulates relational and dynamic properties from a short temporal window of motion. Our initial hypothesis posited that sequential modeling would be critical, leading us to compare static classifiers (MLPs) against temporal models (RNNs). Counter-intuitively, the key discovery occurred when we set the sequence length of a Bidirectional RNN to one (seq_length=1). This modification converted the network's function, compelling it to act as a high-capacity static feature encoder. This architectural change directly led to a significant accuracy improvement, culminating in a final score of 97.60%. Of particular note, our optimized model successfully overcame the most challenging transitional class, 'grabbing', by achieving a balanced F1-score of 0.90. These findings provide a new benchmark for low-level hand-object interaction recognition using structured, interpretable features and lightweight architectures.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 8 of its 20 references that are in the map. First posted on arXiv 10 Dec 2025.
Builds on
8 of 20 references.
References · 8
- 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
- 2021ViViT: A Video Vision Transformercited by 1,620
- 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,209
- 2014Long-term Recurrent Convolutional Networks for Visual Recognition and Descriptioncited by 484
- 2018Human Action Recognition and Prediction: A Surveycited by 227
- 2018Learning Human-Object Interactions by Graph Parsing Neural Networkscited by 176
- 2023A Hierarchical Graph-based Approach for Recognition and Description Generation of Bimanual Actions in Videoscited by 7
- 2023Multi Sentence Description of Complex Manipulation Action Videoscited by 6
Built on by
Not cited yet.