# FusePose: IMU-Vision Sensor Fusion in Kinematic Space for Parametric Human Pose Estimation

arXiv: https://arxiv.org/abs/2208.11960 · HTML: https://mlsanity.com/paper/2208.11960/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 61 references that are in the map.
- First posted on arXiv 25 Aug 2022.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [End-to-end Recovery of Human Shape and Pose](https://mlsanity.com/paper/1712.06584/) (2017)
- [Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image](https://mlsanity.com/paper/1607.08128/) (2016)
- [A simple yet effective baseline for 3d human pose estimation](https://mlsanity.com/paper/1705.03098/) (2017)
- [PARE: Part Attention Regressor for 3D Human Body Estimation](https://mlsanity.com/paper/2104.08527/) (2021)
- [HybrIK: A Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Human Pose and Shape Estimation](https://mlsanity.com/paper/2011.14672/) (2020)
- [Integral Human Pose Regression](https://mlsanity.com/paper/1711.08229/) (2017)
- [Unite the People: Closing the Loop Between 3D and 2D Human Representations](https://mlsanity.com/paper/1701.02468/) (2017)
- [Learnable Triangulation of Human Pose](https://mlsanity.com/paper/1905.05754/) (2019)
- [Human Pose Regression with Residual Log-likelihood Estimation](https://mlsanity.com/paper/2107.11291/) (2021)
- [VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera](https://mlsanity.com/paper/1705.01583/) (2017)
- [Exploiting temporal context for 3D human pose estimation in the wild](https://mlsanity.com/paper/1905.04266/) (2019)
- [TokenPose: Learning Keypoint Tokens for Human Pose Estimation](https://mlsanity.com/paper/2104.03516/) (2021)
- [Cross View Fusion for 3D Human Pose Estimation](https://mlsanity.com/paper/1909.01203/) (2019)
- [Neural Kinematic Networks for Unsupervised Motion Retargetting](https://mlsanity.com/paper/1804.05653/) (2018)
- [Learning Feature Pyramids for Human Pose Estimation](https://mlsanity.com/paper/1708.01101/) (2017)
- [Modeling Human Motion with Quaternion-based Neural Networks](https://mlsanity.com/paper/1901.07677/) (2019)
- [Multi-View Multi-Person 3D Pose Estimation with Plane Sweep Stereo](https://mlsanity.com/paper/2104.02273/) (2021)
- [MotioNet: 3D Human Motion Reconstruction from Monocular Video with Skeleton Consistency](https://mlsanity.com/paper/2006.12075/) (2020)
- [Fusing Wearable IMUs with Multi-View Images for Human Pose Estimation: A Geometric Approach](https://mlsanity.com/paper/2003.11163/) (2020)
- [Feature Boosting Network For 3D Pose Estimation](https://mlsanity.com/paper/1901.04877/) (2019)
- [Revitalizing Optimization for 3D Human Pose and Shape Estimation: A Sparse Constrained Formulation](https://mlsanity.com/paper/2105.13965/) (2021)
- [DeepFuse: An IMU-Aware Network for Real-Time 3D Human Pose Estimation from Multi-View Image](https://mlsanity.com/paper/1912.04071/) (2019)
- [Deep Autoencoder for Combined Human Pose Estimation and body Model Upscaling](https://mlsanity.com/paper/1807.01511/) (2018)

## Built on by (most cited first, 6 of 6)

- [Fusing Monocular Images and Sparse IMU Signals for Real-time Human Motion Capture](https://mlsanity.com/paper/2309.00310/) (2023)
- [Enhancing 3D Human Pose Estimation Amidst Severe Occlusion with Dual Transformer Fusion](https://mlsanity.com/paper/2410.04574/) (2024)
- [Hybrid 3D Human Pose Estimation with Monocular Video and Sparse IMUs](https://mlsanity.com/paper/2404.17837/) (2024)
- [DiffCap: Diffusion-based Real-time Human Motion Capture using Sparse IMUs and a Monocular Camera](https://mlsanity.com/paper/2508.06139/) (2025)
- [AVI-HT: Adaptive Vision-IMU Fusion for 3D Hand Tracking](https://mlsanity.com/paper/2605.21714/) (2026)
- [A Plug-and-Play Multi-Criteria Guidance for Diverse In-Betweening Human Motion Generation](https://mlsanity.com/paper/2508.01590/) (2025)

## Abstract (arXiv)

There exist challenging problems in 3D human pose estimation mission, such as poor performance caused by occlusion and self-occlusion. Recently, IMU-vision sensor fusion is regarded as valuable for solving these problems. However, previous researches on the fusion of IMU and vision data, which is heterogeneous, fail to adequately utilize either IMU raw data or reliable high-level vision features. To facilitate a more efficient sensor fusion, in this work we propose a framework called FusePose under a parametric human kinematic model. Specifically, we aggregate different information of IMU or vision data and introduce three distinctive sensor fusion approaches: NaiveFuse, KineFuse and AdaDeepFuse. NaiveFuse servers as a basic approach that only fuses simplified IMU data and estimated 3D pose in euclidean space. While in kinematic space, KineFuse is able to integrate the calibrated and aligned IMU raw data with converted 3D pose parameters. AdaDeepFuse further develops this kinematical fusion process to an adaptive and end-to-end trainable manner. Comprehensive experiments with ablation studies demonstrate the rationality and superiority of the proposed framework. The performance of 3D human pose estimation is improved compared to the baseline result. On Total Capture dataset, KineFuse surpasses previous state-of-the-art which uses IMU only for testing by 8.6%. AdaDeepFuse surpasses state-of-the-art which uses IMU for both training and testing by 8.5%. Moreover, we validate the generalization capability of our framework through experiments on Human3.6M dataset.