Persistent-Transient Duality: A Multi-mechanism Approach for Modeling Human-Object Interaction
Not on the map. Why?
Abstract (arXiv)
Humans are highly adaptable, swiftly switching between different modes to progressively handle different tasks, situations and contexts. In Human-object interaction (HOI) activities, these modes can be attributed to two mechanisms: (1) the large-scale consistent plan for the whole activity and (2) the small-scale children interactive actions that start and end along the timeline. While neuroscience and cognitive science have confirmed this multi-mechanism nature of human behavior, machine modeling approaches for human motion are trailing behind. While attempted to use gradually morphing structures (e.g., graph attention networks) to model the dynamic HOI patterns, they miss the expeditious and discrete mode-switching nature of the human motion. To bridge that gap, this work proposes to model two concurrent mechanisms that jointly control human motion: the Persistent process that runs continually on the global scale, and the Transient sub-processes that operate intermittently on the local context of the human while interacting with objects. These two mechanisms form an interactive Persistent-Transient Duality that synergistically governs the activity sequences. We model this conceptual duality by a parent-child neural network of Persistent and Transient channels with a dedicated neural module for dynamic mechanism switching. The framework is trialed on HOI motion forecasting. On two rich datasets and a wide variety of settings, the model consistently delivers superior performances, proving its suitability for the challenge.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 27 of its 46 references that are in the map. First posted on arXiv 24 Jul 2023.
Builds on
27 of 46 references.
References · 27
- 2017Attention Is All You Needcited by 60,361
- 2017Graph Attention Networkscited by 8,923
- 2019Heterogeneous Graph Attention Networkcited by 734
- 2017On human motion prediction using recurrent neural networkscited by 338
- 2019Compositionality decomposed: how do neural networks generalise?cited by 265
- 2019Learning Trajectory Dependencies for Human Motion Predictioncited by 246
- 2015Structural-RNN: Deep Learning on Spatio-Temporal Graphscited by 234
- 2020Long-term Human Motion Prediction with Scene Contextcited by 192
- 2018Learning Human-Object Interactions by Graph Parsing Neural Networkscited by 176
- 2020Dynamic Multiscale Graph Neural Networks for 3D Skeleton-Based Human Motion Predictioncited by 132
- 2018Grounded Human-Object Interaction Hotspots from Videocited by 129
- 2020We are More than Our Joints: Predicting how 3D Bodies Movecited by 115
15 more
- 2018Human Motion Prediction via Spatio-Temporal Inpaintingcited by 114
- 2018QuaterNet: A Quaternion-based Recurrent Model for Human Motioncited by 109
- 2021MSR-GCN: Multi-Scale Residual Graph Convolution Networks for Human Motion Predictioncited by 100
- 2022Back to MLP: A Simple Baseline for Human Motion Predictioncited by 100
- 2019Structured Prediction Helps 3D Human Motion Modellingcited by 96
- 2019Context-aware Human Motion Predictioncited by 84
- 2021Space-Time-Separable Graph Convolutional Network for Pose Forecastingcited by 84
- 2017Learning Human Motion Models for Long-term Predictionscited by 67
- 2022Progressively Generating Better Initial Guesses Towards Next Stages for High-Quality Human Motion Predictioncited by 66
- 2018BiHMP-GAN: Bidirectional 3D Human Motion Prediction GANcited by 55
- 2021TRiPOD: Human Trajectory and Pose Dynamics Forecasting in the Wildcited by 52
- 2022MotionMixer: MLP-based 3D Human Body Pose Forecastingcited by 42
- 2019Learning Object-Action Relations from Bimanual Human Demonstration Using Graph Networkscited by 29
- 2018Stacked Spatio-Temporal Graph Convolutional Networks for Action Segmentationcited by 25
- 2021Learning Asynchronous and Sparse Human-Object Interaction in Videoscited by 16
Built on by
Citing papers per year: 2024: 1.
1 paper.