# Lightweight Multi-View 3D Pose Estimation through Camera-Disentangled Representation

arXiv: https://arxiv.org/abs/2004.02186 · HTML: https://mlsanity.com/paper/2004.02186/

## Key facts

- Cited by 57 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Apr 2020.

## Built on by (most cited first, 30 of 57)

- [Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activities](https://mlsanity.com/paper/2203.14712/) (2022)
- [SPEC: Seeing People in the Wild with an Estimated Camera](https://mlsanity.com/paper/2110.00620/) (2021)
- [Direct Multi-view Multi-person 3D Pose Estimation](https://mlsanity.com/paper/2111.04076/) (2021)
- [MotionAGFormer: Enhancing 3D Human Pose Estimation with a Transformer-GCNFormer Network](https://mlsanity.com/paper/2310.16288/) (2023)
- [UmeTrack: Unified multi-view end-to-end hand tracking for VR](https://mlsanity.com/paper/2211.00099/) (2022)
- [Faster VoxelPose: Real-time 3D Human Pose Estimation by Orthographic Projection](https://mlsanity.com/paper/2207.10955/) (2022)
- [Multi-View Multi-Person 3D Pose Estimation with Plane Sweep Stereo](https://mlsanity.com/paper/2104.02273/) (2021)
- [Graph-Based 3D Multi-Person Pose Estimation Using Multi-View Images](https://mlsanity.com/paper/2109.05885/) (2021)
- [Benchmarks and Challenges in Pose Estimation for Egocentric Hand Interactions with Objects](https://mlsanity.com/paper/2403.16428/) (2024)
- [Generalizable Human Pose Triangulation](https://mlsanity.com/paper/2110.00280/) (2021)
- [Shape-aware Multi-Person Pose Estimation from Multi-View Images](https://mlsanity.com/paper/2110.02330/) (2021)
- [SelfPose3d: Self-Supervised Multi-Person Multi-View 3d Pose Estimation](https://mlsanity.com/paper/2404.02041/) (2024)
- [Adaptive Multi-view and Temporal Fusing Transformer for 3D Human Pose Estimation](https://mlsanity.com/paper/2110.05092/) (2021)
- [Fusing Monocular Images and Sparse IMU Signals for Real-time Human Motion Capture](https://mlsanity.com/paper/2309.00310/) (2023)
- [Real-Time Multi-View 3D Human Pose Estimation using Semantic Feedback to Smart Edge Sensors](https://mlsanity.com/paper/2106.14729/) (2021)
- [Weakly-supervised 3D Human Pose Estimation with Cross-view U-shaped Graph Convolutional Network](https://mlsanity.com/paper/2105.10882/) (2021)
- [AirPose: Multi-View Fusion Network for Aerial 3D Human Pose and Shape Estimation](https://mlsanity.com/paper/2201.08093/) (2022)
- [Towards Robust and Smooth 3D Multi-Person Pose Estimation from Monocular Videos in the Wild](https://mlsanity.com/paper/2309.08644/) (2023)
- [On Triangulation as a Form of Self-Supervision for 3D Human Pose Estimation](https://mlsanity.com/paper/2203.15865/) (2022)
- [POEM: Reconstructing Hand in a Point Embedded Multi-view Stereo](https://mlsanity.com/paper/2304.04038/) (2023)
- [Spectral Graphormer: Spectral Graph-based Transformer for Egocentric Two-Hand Reconstruction using Multi-View Color Images](https://mlsanity.com/paper/2308.11015/) (2023)
- [Progressive Multi-view Human Mesh Recovery with Self-Supervision](https://mlsanity.com/paper/2212.05223/) (2022)
- [Markerless Multi-view 3D Human Pose Estimation: a survey](https://mlsanity.com/paper/2407.03817/) (2024)
- [HaMuCo: Hand Pose Estimation via Multiview Collaborative Self-Supervised Learning](https://mlsanity.com/paper/2302.00988/) (2023)
- [Probabilistic Triangulation for Uncalibrated Multi-View 3D Human Pose Estimation](https://mlsanity.com/paper/2309.04756/) (2023)
- [HandDGP: Camera-Space Hand Mesh Prediction with Differentiable Global Positioning](https://mlsanity.com/paper/2407.15844/) (2024)
- [View Consistency Aware Holistic Triangulation for 3D Human Pose Estimation](https://mlsanity.com/paper/2302.11301/) (2023)
- [HumMUSS: Human Motion Understanding using State Space Models](https://mlsanity.com/paper/2404.10880/) (2024)
- [UPose3D: Uncertainty-Aware 3D Human Pose Estimation with Cross-View and Temporal Cues](https://mlsanity.com/paper/2404.14634/) (2024)
- [Multi-view Hand Reconstruction with a Point-Embedded Transformer](https://mlsanity.com/paper/2408.10581/) (2024)

## Abstract (arXiv)

We present a lightweight solution to recover 3D pose from multi-view images captured with spatially calibrated cameras. Building upon recent advances in interpretable representation learning, we exploit 3D geometry to fuse input images into a unified latent representation of pose, which is disentangled from camera view-points. This allows us to reason effectively about 3D pose across different views without using compute-intensive volumetric grids. Our architecture then conditions the learned representation on camera projection operators to produce accurate per-view 2d detections, that can be simply lifted to 3D via a differentiable Direct Linear Transform (DLT) layer. In order to do it efficiently, we propose a novel implementation of DLT that is orders of magnitude faster on GPU architectures than standard SVD-based triangulation methods. We evaluate our approach on two large-scale human pose datasets (H36M and Total Capture): our method outperforms or performs comparably to the state-of-the-art volumetric methods, while, unlike them, yielding real-time performance.