mlsanityData to 14 Oct 2026

Beyond Sequences: A Benchmark for Atomic Hand-Object Interaction Using a Static RNN Encoder

arXiv 2512.09626 ↗First posted 10 Dec 2025cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Reliably predicting human intent in hand-object interactions is an open challenge for computer vision. Our research concentrates on a fundamental sub-problem: the fine-grained classification of atomic interaction states, namely 'approaching', 'grabbing', and 'holding'. To this end, we introduce a structured data engineering process that converts raw videos from the MANIAC dataset into 27,476 statistical-kinematic feature vectors. Each vector encapsulates relational and dynamic properties from a short temporal window of motion. Our initial hypothesis posited that sequential modeling would be critical, leading us to compare static classifiers (MLPs) against temporal models (RNNs). Counter-intuitively, the key discovery occurred when we set the sequence length of a Bidirectional RNN to one (seq_length=1). This modification converted the network's function, compelling it to act as a high-capacity static feature encoder. This architectural change directly led to a significant accuracy improvement, culminating in a final score of 97.60%. Of particular note, our optimized model successfully overcame the most challenging transitional class, 'grabbing', by achieving a balanced F1-score of 0.90. These findings provide a new benchmark for low-level hand-object interaction recognition using structured, interpretable features and lightweight architectures.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 8 of its 20 references that are in the map. First posted on arXiv 10 Dec 2025.

Builds on

8 of 20 references.

References · 8

  1. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  2. 2021ViViT: A Video Vision Transformercited by 1,620
  3. 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,209
  4. 2014Long-term Recurrent Convolutional Networks for Visual Recognition and Descriptioncited by 484
  5. 2018Human Action Recognition and Prediction: A Surveycited by 227
  6. 2018Learning Human-Object Interactions by Graph Parsing Neural Networkscited by 176
  7. 2023A Hierarchical Graph-based Approach for Recognition and Description Generation of Bimanual Actions in Videoscited by 7
  8. 2023Multi Sentence Description of Complex Manipulation Action Videoscited by 6

Built on by

Not cited yet.