Full-Body Motion Reconstruction with Sparse Sensing from Graph Perspective
Not on the map. Why?
Abstract (arXiv)
Estimating 3D full-body pose from sparse sensor data is a pivotal technique employed for the reconstruction of realistic human motions in Augmented Reality and Virtual Reality. However, translating sparse sensor signals into comprehensive human motion remains a challenge since the sparsely distributed sensors in common VR systems fail to capture the motion of full human body. In this paper, we use well-designed Body Pose Graph (BPG) to represent the human body and translate the challenge into a prediction problem of graph missing nodes. Then, we propose a novel full-body motion reconstruction framework based on BPG. To establish BPG, nodes are initially endowed with features extracted from sparse sensor signals. Features from identifiable joint nodes across diverse sensors are amalgamated and processed from both temporal and spatial perspectives. Temporal dynamics are captured using the Temporal Pyramid Structure, while spatial relations in joint movements inform the spatial attributes. The resultant features serve as the foundational elements of the BPG nodes. To further refine the BPG, node features are updated through a graph neural network that incorporates edge reflecting varying joint relations. Our method's effectiveness is evidenced by the attained state-of-the-art performance, particularly in lower body motion, outperforming other baseline methods. Additionally, an ablation study validates the efficacy of each module in our proposed framework.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 27 of its 45 references that are in the map. First posted on arXiv 22 Jan 2024.
Builds on
27 of 45 references.
References · 27
- 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
- 2017Graph Attention Networkscited by 8,923
- 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,612
- 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,236
- 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,209
- 2018Link Prediction Based on Graph Neural Networkscited by 832
- 2017On human motion prediction using recurrent neural networkscited by 338
- 2021SCINet: Time Series Modeling and Forecasting with Sample Convolution and Interactioncited by 338
- 2019Actional-Structural Graph Convolutional Networks for Skeleton-based Action Recognitioncited by 228
- 2019Semantic Graph Convolutional Networks for 3D Human Pose Regressioncited by 204
- 2019An Attention Enhanced Graph Convolutional LSTM Network for Skeleton-Based Action Recognitioncited by 151
- 2022Physical Inertial Poser (PIP): Physics-aware Real-time Human Motion Tracking from Sparse Inertial Sensorscited by 146
15 more
- 2020Dynamic Multiscale Graph Neural Networks for 3D Skeleton-Based Human Motion Predictioncited by 132
- 2022AvatarPoser: Articulated Full-Body Pose Tracking from Sparse Motion Sensingcited by 110
- 2017Sparse Inertial Poser: Automatic 3D Human Pose Estimation from Sparse IMUscited by 109
- 2023Avatars Grow Legs: Generating Smooth Human Motion from Sparse Tracking Inputs with Diffusion Modelcited by 107
- 2019Skeleton-Based Action Recognition with Multi-Stream Adaptive Graph Convolutional Networkscited by 103
- 2021Space-Time-Separable Graph Convolutional Network for Pose Forecastingcited by 84
- 2020Dynamic GCN: Context-enriched Topology Learning for Skeleton-based Action Recognitioncited by 82
- 2022QuestSim: Human Motion Tracking from Sparse Sensors with Simulated Avatarscited by 77
- 2021On the Unreasonable Effectiveness of Feature propagation in Learning on Graphs with Missing Node Featurescited by 63
- 2021Learning Skeletal Graph Neural Networks for Hard 3D Pose Estimationcited by 62
- 2020Graph Geometry Interaction Learningcited by 55
- 2020Graph Convolutional Networks for Graphs Containing Missing Featurescited by 47
- 2021LoBSTr: Real-time Lower-body Pose Prediction from Sparse Upper-body Tracking Signalscited by 45
- 2020Incomplete Graph Representation and Learning via Partial Graph Neural Networkscited by 16
- 2022Non-Local Latent Relation Distillation for Self-Adaptive 3D Human Pose Estimationcited by 6
Built on by
Citing papers per year: 2025: 1.
1 paper.