# Fusing Wearable IMUs with Multi-View Images for Human Pose Estimation: A Geometric Approach

arXiv: https://arxiv.org/abs/2003.11163 · HTML: https://mlsanity.com/paper/2003.11163/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Mar 2020.

## Built on by (most cited first, 27 of 27)

- [EMDB: The Electromagnetic Database of Global 3D Human Pose and Shape in the Wild](https://mlsanity.com/paper/2308.16894/) (2023)
- [MM-Fi: Multi-Modal Non-Intrusive 4D Human Dataset for Versatile Wireless Sensing](https://mlsanity.com/paper/2305.10345/) (2023)
- [Transformer Inertial Poser: Real-time Human Motion Reconstruction from Sparse IMUs with Simultaneous Terrain Generation](https://mlsanity.com/paper/2203.15720/) (2022)
- [GFPose: Learning 3D Human Pose Prior with Gradient Fields](https://mlsanity.com/paper/2212.08641/) (2022)
- [LiDAR-aid Inertial Poser: Large-scale Human Motion Capture by Sparse Inertial and LiDAR Sensors](https://mlsanity.com/paper/2205.15410/) (2022)
- [I'M HOI: Inertia-aware Monocular Capture of 3D Human-Object Interactions](https://mlsanity.com/paper/2312.08869/) (2023)
- [Faster VoxelPose: Real-time 3D Human Pose Estimation by Orthographic Projection](https://mlsanity.com/paper/2207.10955/) (2022)
- [Dynamic Inertial Poser (DynaIP): Part-Based Motion Dynamics Learning for Enhanced Human Pose Estimation with Sparse Inertial Sensors](https://mlsanity.com/paper/2312.02196/) (2023)
- [TransPose: Real-time 3D Human Translation and Pose Estimation with Six Inertial Sensors](https://mlsanity.com/paper/2105.04605/) (2021)
- [Vision-Based Human Pose Estimation via Deep Learning: A Survey](https://mlsanity.com/paper/2308.13872/) (2023)
- [Adaptive Multi-view and Temporal Fusing Transformer for 3D Human Pose Estimation](https://mlsanity.com/paper/2110.05092/) (2021)
- [HybridCap: Inertia-aid Monocular Capture of Challenging Human Motions](https://mlsanity.com/paper/2203.09287/) (2022)
- [Fusing Monocular Images and Sparse IMU Signals for Real-time Human Motion Capture](https://mlsanity.com/paper/2309.00310/) (2023)
- [Ultra Inertial Poser: Scalable Motion Capture and Tracking from Sparse Inertial Sensors and Ultra-Wideband Ranging](https://mlsanity.com/paper/2404.19541/) (2024)
- [FLEX: Extrinsic Parameters-free Multi-view 3D Human Motion Reconstruction](https://mlsanity.com/paper/2105.01937/) (2021)
- [VirtualPose: Learning Generalizable 3D Human Pose Models from Virtual Data](https://mlsanity.com/paper/2207.09949/) (2022)
- [FusePose: IMU-Vision Sensor Fusion in Kinematic Space for Parametric Human Pose Estimation](https://mlsanity.com/paper/2208.11960/) (2022)
- [View Consistency Aware Holistic Triangulation for 3D Human Pose Estimation](https://mlsanity.com/paper/2302.11301/) (2023)
- [Human Pose-based Estimation, Tracking and Action Recognition with Deep Learning: A Survey](https://mlsanity.com/paper/2310.13039/) (2023)
- [Human Mesh Recovery from Arbitrary Multi-view Images](https://mlsanity.com/paper/2403.12434/) (2024)
- [FreeCap: Hybrid Calibration-Free Motion Capture in Open Environments](https://mlsanity.com/paper/2411.04469/) (2024)
- [Multimodal Active Measurement for Human Mesh Recovery in Close Proximity](https://mlsanity.com/paper/2310.08116/) (2023)
- [DiffCap: Diffusion-based Real-time Human Motion Capture using Sparse IMUs and a Monocular Camera](https://mlsanity.com/paper/2508.06139/) (2025)
- [Stereo-Inertial Poser: Towards Metric-Accurate Shape-Aware Motion Capture Using Sparse IMUs and a Single Stereo Camera](https://mlsanity.com/paper/2603.02130/) (2026)
- [AVI-HT: Adaptive Vision-IMU Fusion for 3D Hand Tracking](https://mlsanity.com/paper/2605.21714/) (2026)
- [Modeling the Uncertainty with Maximum Discrepant Students for Semi-supervised 2D Pose Estimation](https://mlsanity.com/paper/2311.01770/) (2023)
- [IMU-HOI: A Symbiotic Framework for Coherent Human-Object Interaction and Motion Capture via Contact-Conscious Inertial Fusion](https://mlsanity.com/paper/2606.28604/) (2026)

## Abstract (arXiv)

We propose to estimate 3D human pose from multi-view images and a few IMUs attached at person's limbs. It operates by firstly detecting 2D poses from the two signals, and then lifting them to the 3D space. We present a geometric approach to reinforce the visual features of each pair of joints based on the IMUs. This notably improves 2D pose estimation accuracy especially when one joint is occluded. We call this approach Orientation Regularized Network (ORN). Then we lift the multi-view 2D poses to the 3D space by an Orientation Regularized Pictorial Structure Model (ORPSM) which jointly minimizes the projection error between the 3D and 2D poses, along with the discrepancy between the 3D pose and IMU orientations. The simple two-step approach reduces the error of the state-of-the-art by a large margin on a public dataset. Our code will be released at https://github.com/CHUNYUWANG/imu-human-pose-pytorch.