# DeepFuse: An IMU-Aware Network for Real-Time 3D Human Pose Estimation from Multi-View Image

arXiv: https://arxiv.org/abs/1912.04071 · HTML: https://mlsanity.com/paper/1912.04071/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Dec 2019.

## Built on by (most cited first, 14 of 14)

- [HuMMan: Multi-Modal 4D Human Dataset for Versatile Sensing and Modeling](https://mlsanity.com/paper/2204.13686/) (2022)
- [Vision-Based Human Pose Estimation via Deep Learning: A Survey](https://mlsanity.com/paper/2308.13872/) (2023)
- [Adaptive Multi-view and Temporal Fusing Transformer for 3D Human Pose Estimation](https://mlsanity.com/paper/2110.05092/) (2021)
- [Fusing Monocular Images and Sparse IMU Signals for Real-time Human Motion Capture](https://mlsanity.com/paper/2309.00310/) (2023)
- [FLEX: Extrinsic Parameters-free Multi-view 3D Human Motion Reconstruction](https://mlsanity.com/paper/2105.01937/) (2021)
- [FusePose: IMU-Vision Sensor Fusion in Kinematic Space for Parametric Human Pose Estimation](https://mlsanity.com/paper/2208.11960/) (2022)
- [Multimodal Active Measurement for Human Mesh Recovery in Close Proximity](https://mlsanity.com/paper/2310.08116/) (2023)
- [Towards Multi-Modal Animal Pose Estimation: A Survey and In-Depth Analysis](https://mlsanity.com/paper/2410.09312/) (2024)
- [DiffCap: Diffusion-based Real-time Human Motion Capture using Sparse IMUs and a Monocular Camera](https://mlsanity.com/paper/2508.06139/) (2025)
- [ConvFormer: Parameter Reduction in Transformer Models for 3D Human Pose Estimation by Leveraging Dynamic Multi-Headed Convolutional Attention](https://mlsanity.com/paper/2304.02147/) (2023)
- [WiFlow: A Lightweight WiFi-based Continuous Human Pose Estimation Network with Spatio-Temporal Feature Decoupling](https://mlsanity.com/paper/2602.08661/) (2026)
- [Unconstrained Multi-view Human Pose Estimation with Algebraic Priors](https://mlsanity.com/paper/2604.24312/) (2026)
- [One Sensor, Whole Body - 3D Body Pose from a Single Consumer Earbud IMU](https://mlsanity.com/paper/2609.34978/) (2026)
- [Reliability-Gated Fusion of Consumer Head and Foot IMUs for Lower-Body 3D Pose](https://mlsanity.com/paper/2609.35764/) (2026)

## Abstract (arXiv)

In this paper, we propose a two-stage fully 3D network, namely DeepFuse, to estimate human pose in 3D space by fusing body-worn Inertial Measurement Unit (IMU) data and multi-view images deeply. The first stage is designed for pure vision estimation. To preserve data primitiveness of multi-view inputs, the vision stage uses multi-channel volume as data representation and 3D soft-argmax as activation layer. The second one is the IMU refinement stage which introduces an IMU-bone layer to fuse the IMU and vision data earlier at data level. without requiring a given skeleton model a priori, we can achieve a mean joint error of 28.9mm on TotalCapture dataset and 13.4mm on Human3.6M dataset under protocol 1, improving the SOTA result by a large margin. Finally, we discuss the effectiveness of a fully 3D network for 3D pose estimation experimentally which may benefit future research.