mlsanityData to 14 Oct 2026

Learn to Predict How Humans Manipulate Large-sized Objects from Interactive Motions

arXiv 2206.12612 ↗First posted 25 Jun 2022cs.CV cs.HCcited by 21

Not on the map. Why?

Abstract (arXiv)

Understanding human intentions during interactions has been a long-lasting theme, that has applications in human-robot interaction, virtual reality and surveillance. In this study, we focus on full-body human interactions with large-sized daily objects and aim to predict the future states of objects and humans given a sequential observation of human-object interaction. As there is no such dataset dedicated to full-body human interactions with large-sized daily objects, we collected a large-scale dataset containing thousands of interactions for training and evaluation purposes. We also observe that an object's intrinsic physical properties are useful for the object motion prediction, and thus design a set of object dynamic descriptors to encode such intrinsic properties. We treat the object dynamic descriptors as a new modality and propose a graph neural network, HO-GCN, to fuse motion data and dynamic descriptors for the prediction task. We show the proposed network that consumes dynamic descriptors can achieve state-of-the-art prediction results and help the network better generalize to unseen objects. We also demonstrate the predicted results are useful for human-robot collaborations.

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). Builds on 21 of its 41 references that are in the map. First posted on arXiv 25 Jun 2022.

Builds on

21 of 41 references.

References · 21

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,209
  3. 2016NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysiscited by 686
  4. 2019NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understandingcited by 533
  5. 2016Interaction Networks for Learning about Objects, Relations and Physicscited by 509
  6. 2018Gibson Env: Real-World Perception for Embodied Agentscited by 481
  7. 2020GRAB: A Dataset of Whole-Body Human Grasping of Objectscited by 405
  8. 2016Unsupervised Learning for Physical Interaction through Video Predictioncited by 260
  9. 2017First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotationscited by 230
  10. 2019Actional-Structural Graph Convolutional Networks for Skeleton-based Action Recognitioncited by 228
  11. 2020Long-term Human Motion Prediction with Scene Contextcited by 192
  12. 2018Co-occurrence Feature Learning from Skeleton Data for Action Recognition and Detection with Hierarchical Aggregationcited by 106
9 more
  1. 2017View Adaptive Recurrent Neural Networks for High Performance Human Action Recognition from Skeleton Datacited by 105
  2. 2017Deep representation learning for human motion prediction and classificationcited by 98
  3. 2019Context-aware Human Motion Predictioncited by 84
  4. 2018Reasoning About Physical Interactions with Object-Oriented Prediction and Planningcited by 51
  5. 2019Estimating 3D Motion and Forces of Person-Object Interactions from Monocular Videocited by 48
  6. 2018Skeleton-Based Action Recognition with Spatial Reasoning and Temporal Stack Learningcited by 47
  7. 2020Use the Force, Luke! Learning to Predict Physical Forces by Simulating Effectscited by 40
  8. 2017Investigation of Different Skeleton Features for CNN-based 3D Action Recognitioncited by 12
  9. 2019Skeleton-based Action Recognition of People Handling Objectscited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 6; 2024: 7; 2025: 7; 2026: 1.

21 papers.

Sort:

By area:

  1. 2023InterDiff: Generating 3D Human-Object Interactions with Physics-Informed Diffusioncited by 182
  2. 2023CG-HOI: Contact-Guided 3D Human-Object Interaction Generationcited by 138
  3. 2023Controllable Human-Object Interaction Synthesiscited by 109
  4. 2023TLControl: Trajectory and Language Control for Human Motion Synthesiscited by 94
  5. 2023EMDM: Efficient Motion Diffusion Model for Fast and High-Quality Motion Generationcited by 89
  6. 2024TACO: Benchmarking Generalizable Bimanual Tool-ACtion-Object Understandingcited by 88
  7. 2025InterMimic: Towards Universal Whole-Body Control for Physics-Based Human-Object Interactionscited by 72
  8. 2024InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object Interactioncited by 64
  9. 2024CORE4D: A 4D Human-Object-Human Interaction Dataset for Collaborative Object REarrangementcited by 44
  10. 2023Object pop-up: Can we infer 3D objects and their poses from human interactions alone?cited by 37
  11. 2025InterAct: Advancing Large-Scale Versatile 3D Human-Object Interaction Generationcited by 32
  12. 2025HUMOTO: A 4D Dataset of Mocap Human Object Interactionscited by 31
9 more
  1. 2024ZeroHSI: Zero-Shot 4D Human-Scene Interaction by Video Generationcited by 25
  2. 2024MotionWavelet: Human Motion Prediction via Wavelet Manifold Learningcited by 8
  3. 2024FORCE: Physics-aware Human-object Interactioncited by 7
  4. 20253D Human Interaction Generation: A Surveycited by 7
  5. 2025Zero-Shot Human-Object Interaction Synthesis with Multimodal Priorscited by 7
  6. 2025HOI-PAGE: Zero-Shot Human-Object Interaction Generation with Part Affordance Guidancecited by 6
  7. 2024COLLAGE: Collaborative Human-Agent Interaction Generation using Hierarchical Latent Diffusion and Language Modelscited by 3
  8. 2025CoopDiff: Anticipating 3D Human-object Interactions via Contact-consistent Decoupled Diffusioncited by 0
  9. 2026SynAgent: Generalizable Cooperative Humanoid Manipulation via Solo-to-Cooperative Agent Synergycited by 0