# DECA: Deep viewpoint-Equivariant human pose estimation using Capsule Autoencoders arXiv: https://arxiv.org/abs/2108.08557 ยท HTML: https://mlsanity.com/paper/2108.08557/ ## Key facts - Cited by 23 papers in the map (data through 14 Oct 2026). - Builds on 20 of its 40 references that are in the map. - First posted on arXiv 19 Aug 2021. ## Builds on (most cited first, 20 of 20) - [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017) - [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016) - [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016) - [Dynamic Routing Between Capsules](https://mlsanity.com/paper/1710.09829/) (2017) - [Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loop](https://mlsanity.com/paper/1909.12828/) (2019) - [VIBE: Video Inference for Human Body Pose and Shape Estimation](https://mlsanity.com/paper/1912.05656/) (2019) - [Spherical CNNs](https://mlsanity.com/paper/1801.10130/) (2018) - [Panoptic Studio: A Massively Multiview System for Social Interaction Capture](https://mlsanity.com/paper/1612.03153/) (2016) - [Human Pose Estimation with Iterative Error Feedback](https://mlsanity.com/paper/1507.06550/) (2015) - [V2V-PoseNet: Voxel-to-Voxel Prediction Network for Accurate 3D Hand and Human Pose Estimation from a Single Depth Map](https://mlsanity.com/paper/1711.07399/) (2017) - [LCR-Net++: Multi-person 2D and 3D Pose Detection in Natural Images](https://mlsanity.com/paper/1803.00455/) (2018) - [Stacked Capsule Autoencoders](https://mlsanity.com/paper/1906.06818/) (2019) - [Lifting from the Deep: Convolutional 3D Pose Estimation from a Single Image](https://mlsanity.com/paper/1701.00295/) (2017) - [A2J: Anchor-to-Joint Regression Network for 3D Articulated Pose Estimation from a Single Depth Image](https://mlsanity.com/paper/1908.09999/) (2019) - [Sparseness Meets Deepness: 3D Human Pose Estimation from Monocular Video](https://mlsanity.com/paper/1511.09439/) (2015) - [Capsule Routing via Variational Bayes](https://mlsanity.com/paper/1905.11455/) (2019) - [Feature Boosting Network For 3D Pose Estimation](https://mlsanity.com/paper/1901.04877/) (2019) - [3D Human Pose Machines with Self-supervised Learning](https://mlsanity.com/paper/1901.03798/) (2019) - [Towards Good Practices for Deep 3D Hand Pose Estimation](https://mlsanity.com/paper/1707.07248/) (2017) - [Human Pose Estimation from Depth Images via Inference Embedded Multi-task Learning](https://mlsanity.com/paper/1608.03932/) (2016) ## Built on by (most cited first, 23 of 23) - [2D Human Pose Estimation: A Survey](https://mlsanity.com/paper/2204.07370/) (2022) - [GazeOnce: Real-Time Multi-Person Gaze Estimation](https://mlsanity.com/paper/2204.09480/) (2022) - [LiDAR-HMR: 3D Human Mesh Recovery from LiDAR](https://mlsanity.com/paper/2311.11971/) (2023) - [PointHPS: Cascaded 3D Human Pose and Shape Estimation from Point Clouds](https://mlsanity.com/paper/2308.14492/) (2023) - [Lightweight Super-Resolution Head for Human Pose Estimation](https://mlsanity.com/paper/2307.16765/) (2023) - [GTPT: Group-based Token Pruning Transformer for Efficient Human Pose Estimation](https://mlsanity.com/paper/2407.10756/) (2024) - [Human Pose Estimation in Monocular Omnidirectional Top-View Images](https://mlsanity.com/paper/2304.08186/) (2023) - [Applications of Deep Learning for Top-View Omnidirectional Imaging: A Survey](https://mlsanity.com/paper/2304.08193/) (2023) - [ACRNet: Attention Cube Regression Network for Multi-view Real-time 3D Human Pose Estimation in Telemedicine](https://mlsanity.com/paper/2210.05130/) (2022) - [NToP: NeRF-Powered Large-scale Dataset Generation for 2D and 3D Human Pose Estimation in Top-View Fisheye Images](https://mlsanity.com/paper/2402.18196/) (2024) - [Sports Analysis and VR Viewing System Based on Player Tracking and Pose Estimation with Multimodal and Multiview Sensors](https://mlsanity.com/paper/2405.01112/) (2024) - [SPiKE: 3D Human Pose from Point Cloud Sequences](https://mlsanity.com/paper/2409.01879/) (2024) - [Perspective Transformation Layer](https://mlsanity.com/paper/2201.05706/) (2022) - [LiCamPose: Combining Multi-View LiDAR and RGB Cameras for Robust Single-timestamp 3D Human Pose Estimation](https://mlsanity.com/paper/2312.06409/) (2023) - [PixelSmile: Toward Fine-Grained Facial Expression Editing](https://mlsanity.com/paper/2603.25728/) (2026) - [Iterative collaborative routing among equivariant capsules for transformation-robust capsule networks](https://mlsanity.com/paper/2210.11095/) (2022) - [Capsules as viewpoint learners for human pose estimation](https://mlsanity.com/paper/2302.06194/) (2023) - [3D Pose Nowcasting: Forecast the Future to Improve the Present](https://mlsanity.com/paper/2308.12914/) (2023) - [Towards AI enabled automated tracking of multiple boxers](https://mlsanity.com/paper/2311.11471/) (2023) - [A Unified Simulation Framework for Visual and Behavioral Fidelity in Crowd Analysis](https://mlsanity.com/paper/2312.02613/) (2023) - [Point2Pose: A Generative Framework for 3D Human Pose Estimation with Multi-View Point Cloud Dataset](https://mlsanity.com/paper/2512.10321/) (2025) - [FlyPose: Towards Robust Human Pose Estimation From Aerial Views](https://mlsanity.com/paper/2601.05747/) (2026) - [On the Role of Rotation Equivariance in Monocular 2D-to-3D Human Pose Lifting](https://mlsanity.com/paper/2601.13913/) (2026) ## Abstract (arXiv) Human Pose Estimation (HPE) aims at retrieving the 3D position of human joints from images or videos. We show that current 3D HPE methods suffer a lack of viewpoint equivariance, namely they tend to fail or perform poorly when dealing with viewpoints unseen at training time. Deep learning methods often rely on either scale-invariant, translation-invariant, or rotation-invariant operations, such as max-pooling. However, the adoption of such procedures does not necessarily improve viewpoint generalization, rather leading to more data-dependent methods. To tackle this issue, we propose a novel capsule autoencoder network with fast Variational Bayes capsule routing, named DECA. By modeling each joint as a capsule entity, combined with the routing algorithm, our approach can preserve the joints' hierarchical and geometrical structure in the feature space, independently from the viewpoint. By achieving viewpoint equivariance, we drastically reduce the network data dependency at training time, resulting in an improved ability to generalize for unseen viewpoints. In the experimental validation, we outperform other methods on depth images from both seen and unseen viewpoints, both top-view, and front-view. In the RGB domain, the same network gives state-of-the-art results on the challenging viewpoint transfer task, also establishing a new framework for top-view HPE. The code can be found at https://github.com/mmlab-cv/DECA.