# Geometry-Biased Transformer for Robust Multi-View 3D Human Pose Reconstruction

arXiv: https://arxiv.org/abs/2312.17106 · HTML: https://mlsanity.com/paper/2312.17106/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 44 references that are in the map.
- First posted on arXiv 28 Dec 2023.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [AMASS: Archive of Motion Capture as Surface Shapes](https://mlsanity.com/paper/1904.03278/) (2019)
- [YOLOX: Exceeding YOLO Series in 2021](https://mlsanity.com/paper/2107.08430/) (2021)
- [Perceiver IO: A General Architecture for Structured Inputs & Outputs](https://mlsanity.com/paper/2107.14795/) (2021)
- [End-to-End Human Pose and Mesh Reconstruction with Transformers](https://mlsanity.com/paper/2012.09760/) (2020)
- [3D human pose estimation in video with temporal convolutions and semi-supervised training](https://mlsanity.com/paper/1811.11742/) (2018)
- [A simple yet effective baseline for 3d human pose estimation](https://mlsanity.com/paper/1705.03098/) (2017)
- [Light Field Networks: Neural Scene Representations with Single-Evaluation Rendering](https://mlsanity.com/paper/2106.02634/) (2021)
- [Panoptic Studio: A Massively Multiview System for Social Interaction Capture](https://mlsanity.com/paper/1612.03153/) (2016)
- [Learnable Triangulation of Human Pose](https://mlsanity.com/paper/1905.05754/) (2019)
- [MixSTE: Seq2seq Mixed Spatio-Temporal Encoder for 3D Human Pose Estimation in Video](https://mlsanity.com/paper/2203.00859/) (2022)
- [TransPose: Keypoint Localization via Transformer](https://mlsanity.com/paper/2012.14214/) (2020)
- [Epipolar Transformers](https://mlsanity.com/paper/2005.04551/) (2020)
- [Cross View Fusion for 3D Human Pose Estimation](https://mlsanity.com/paper/1909.01203/) (2019)
- [Probabilistic Monocular 3D Human Pose Estimation with Normalizing Flows](https://mlsanity.com/paper/2107.13788/) (2021)
- [3D Human Pose Estimation from a Single Image via Distance Matrix Regression](https://mlsanity.com/paper/1611.09010/) (2016)
- [Learning Monocular 3D Human Pose Estimation from Multi-view Images](https://mlsanity.com/paper/1803.04775/) (2018)
- [Lightweight Multi-View 3D Pose Estimation through Camera-Disentangled Representation](https://mlsanity.com/paper/2004.02186/) (2020)
- [Harvesting Multiple Views for Marker-less 3D Human Pose Annotations](https://mlsanity.com/paper/1704.04793/) (2017)
- [Poseur: Direct Human Pose Regression with Transformers](https://mlsanity.com/paper/2201.07412/) (2022)
- [AdaFuse: Adaptive Multiview Fusion for Accurate Human Pose Estimation in the Wild](https://mlsanity.com/paper/2010.13302/) (2020)
- [TransFusion: Cross-view Fusion with Transformer for 3D Human Pose Estimation](https://mlsanity.com/paper/2110.09554/) (2021)
- [PatchDropout: Economizing Vision Transformers Using Patch Dropout](https://mlsanity.com/paper/2208.07220/) (2022)
- [PPT: token-Pruned Pose Transformer for monocular and multi-view human pose estimation](https://mlsanity.com/paper/2209.08194/) (2022)
- [How Robust is 3D Human Pose Estimation to Occlusion?](https://mlsanity.com/paper/1808.09316/) (2018)
- [MetaFuse: A Pre-trained Fusion Model for Human Pose Estimation](https://mlsanity.com/paper/2003.13239/) (2020)

## Built on by (most cited first, 2 of 2)

- [Multi-view Pose Fusion for Occlusion-Aware 3D Human Pose Estimation](https://mlsanity.com/paper/2408.15810/) (2024)
- [SkelSplat: Robust Multi-view 3D Human Pose Estimation with Differentiable Gaussian Rendering](https://mlsanity.com/paper/2511.08294/) (2025)

## Abstract (arXiv)

We address the challenges in estimating 3D human poses from multiple views under occlusion and with limited overlapping views. We approach multi-view, single-person 3D human pose reconstruction as a regression problem and propose a novel encoder-decoder Transformer architecture to estimate 3D poses from multi-view 2D pose sequences. The encoder refines 2D skeleton joints detected across different views and times, fusing multi-view and temporal information through global self-attention. We enhance the encoder by incorporating a geometry-biased attention mechanism, effectively leveraging geometric relationships between views. Additionally, we use detection scores provided by the 2D pose detector to further guide the encoder's attention based on the reliability of the 2D detections. The decoder subsequently regresses the 3D pose sequence from these refined tokens, using pre-defined queries for each joint. To enhance the generalization of our method to unseen scenes and improve resilience to missing joints, we implement strategies including scene centering, synthetic views, and token dropout. We conduct extensive experiments on three benchmark public datasets, Human3.6M, CMU Panoptic and Occlusion-Persons. Our results demonstrate the efficacy of our approach, particularly in occluded scenes and when few views are available, which are traditionally challenging scenarios for triangulation-based methods.