# TransFusion: Cross-view Fusion with Transformer for 3D Human Pose Estimation

arXiv: https://arxiv.org/abs/2110.09554 · HTML: https://mlsanity.com/paper/2110.09554/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 50 of its 67 references that are in the map.
- First posted on arXiv 18 Oct 2021.

## Builds on (most cited first, 30 of 50)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [LXMERT: Learning Cross-Modality Encoder Representations from Transformers](https://mlsanity.com/paper/1908.07490/) (2019)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision](https://mlsanity.com/paper/2102.03334/) (2021)
- [VL-BERT: Pre-training of Generic Visual-Linguistic Representations](https://mlsanity.com/paper/1908.08530/) (2019)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification](https://mlsanity.com/paper/2103.14899/) (2021)
- [Simple Baselines for Human Pose Estimation and Tracking](https://mlsanity.com/paper/1804.06208/) (2018)
- [Unicoder-VL: A Universal Encoder for Vision and Language by Cross-modal Pre-training](https://mlsanity.com/paper/1908.06066/) (2019)
- [End-to-End Video Instance Segmentation with Transformers](https://mlsanity.com/paper/2011.14503/) (2020)
- [End-to-End Human Pose and Mesh Reconstruction with Transformers](https://mlsanity.com/paper/2012.09760/) (2020)
- [Cascaded Pyramid Network for Multi-Person Pose Estimation](https://mlsanity.com/paper/1711.07319/) (2017)
- [Convolutional Pose Machines](https://mlsanity.com/paper/1602.00134/) (2016)
- [3D Human Pose Estimation with Spatial and Temporal Transformers](https://mlsanity.com/paper/2103.10455/) (2021)
- [FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape from Single RGB Images](https://mlsanity.com/paper/1909.04349/) (2019)
- [Hand Keypoint Detection in Single Images using Multiview Bootstrapping](https://mlsanity.com/paper/1704.07809/) (2017)
- [3D Hand Shape and Pose Estimation from a Single RGB Image](https://mlsanity.com/paper/1903.00812/) (2019)
- [Learnable Triangulation of Human Pose](https://mlsanity.com/paper/1905.05754/) (2019)
- [Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation](https://mlsanity.com/paper/1406.2984/) (2014)
- [VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera](https://mlsanity.com/paper/1705.01583/) (2017)
- [End-to-end Hand Mesh Recovery from a Monocular RGB Image](https://mlsanity.com/paper/1902.09305/) (2019)

## Built on by (most cited first, 30 of 32)

- [AFTer-UNet: Axial Fusion Transformer UNet for Medical Image Segmentation](https://mlsanity.com/paper/2110.10403/) (2021)
- [TORE: Token Reduction for Efficient Human Mesh Recovery with Transformer](https://mlsanity.com/paper/2211.10705/) (2022)
- [PPT: token-Pruned Pose Transformer for monocular and multi-view human pose estimation](https://mlsanity.com/paper/2209.08194/) (2022)
- [3D Human Pose Perception from Egocentric Stereo Videos](https://mlsanity.com/paper/2401.00889/) (2023)
- [Identity-Aware Hand Mesh Estimation and Personalization from RGB Images](https://mlsanity.com/paper/2209.10840/) (2022)
- [On Triangulation as a Form of Self-Supervision for 3D Human Pose Estimation](https://mlsanity.com/paper/2203.15865/) (2022)
- [GeoFill: Reference-Based Image Inpainting with Better Geometric Understanding](https://mlsanity.com/paper/2201.08131/) (2022)
- [The Best of Both Worlds: Combining Model-based and Nonparametric Approaches for 3D Human Body Estimation](https://mlsanity.com/paper/2205.00508/) (2022)
- [Mitigating Motion Blur for Robust 3D Baseball Player Pose Modeling for Pitch Analysis](https://mlsanity.com/paper/2309.01010/) (2023)
- [Probabilistic Triangulation for Uncalibrated Multi-View 3D Human Pose Estimation](https://mlsanity.com/paper/2309.04756/) (2023)
- [FreeMan: Towards Benchmarking 3D Human Pose Estimation under Real-World Conditions](https://mlsanity.com/paper/2309.05073/) (2023)
- [Bring Your Rear Cameras for Egocentric 3D Human Pose Estimation](https://mlsanity.com/paper/2503.11652/) (2025)
- [View Consistency Aware Holistic Triangulation for 3D Human Pose Estimation](https://mlsanity.com/paper/2302.11301/) (2023)
- [Human Mesh Recovery from Arbitrary Multi-view Images](https://mlsanity.com/paper/2403.12434/) (2024)
- [UPose3D: Uncertainty-Aware 3D Human Pose Estimation with Cross-View and Temporal Cues](https://mlsanity.com/paper/2404.14634/) (2024)
- [Multi-view Pose Fusion for Occlusion-Aware 3D Human Pose Estimation](https://mlsanity.com/paper/2408.15810/) (2024)
- [Uncertainty Estimation for 3D Object Detection via Evidential Learning](https://mlsanity.com/paper/2410.23910/) (2024)
- [Kinematic-aware Hierarchical Attention Network for Human Pose Estimation in Videos](https://mlsanity.com/paper/2211.15868/) (2022)
- [Learning to Estimate External Forces of Human Motion in Video](https://mlsanity.com/paper/2207.05845/) (2022)
- [V-VIPE: Variational View Invariant Pose Embedding](https://mlsanity.com/paper/2407.07092/) (2024)
- [Temporal Transformer Networks with Self-Supervision for Action Recognition](https://mlsanity.com/paper/2112.07338/) (2021)
- [Geometry-Biased Transformer for Robust Multi-View 3D Human Pose Reconstruction](https://mlsanity.com/paper/2312.17106/) (2023)
- [An uncertainty-aware framework for data-efficient multi-view animal pose estimation](https://mlsanity.com/paper/2510.09903/) (2025)
- [RUMPL: Ray-Based Transformers for Universal Multi-View 2D to 3D Human Pose Lifting](https://mlsanity.com/paper/2512.15488/) (2025)
- [Epipolar Attention Field Transformers for Bird's Eye View Semantic Segmentation](https://mlsanity.com/paper/2412.01595/) (2024)
- [Unconstrained Multi-view Human Pose Estimation with Algebraic Priors](https://mlsanity.com/paper/2604.24312/) (2026)
- [Distilling Token-Pruned Pose Transformer for 2D Human Pose Estimation](https://mlsanity.com/paper/2304.05548/) (2023)
- [Occlusion Resilient 3D Human Pose Estimation](https://mlsanity.com/paper/2402.11036/) (2024)
- [Self-learning Canonical Space for Multi-view 3D Human Pose Estimation](https://mlsanity.com/paper/2403.12440/) (2024)
- [Physically Plausible Human-Object Rendering from Sparse Views via 3D Gaussian Splatting](https://mlsanity.com/paper/2503.09640/) (2025)

## Abstract (arXiv)

Estimating the 2D human poses in each view is typically the first step in calibrated multi-view 3D pose estimation. But the performance of 2D pose detectors suffers from challenging situations such as occlusions and oblique viewing angles. To address these challenges, previous works derive point-to-point correspondences between different views from epipolar geometry and utilize the correspondences to merge prediction heatmaps or feature representations. Instead of post-prediction merge/calibration, here we introduce a transformer framework for multi-view 3D pose estimation, aiming at directly improving individual 2D predictors by integrating information from different views. Inspired by previous multi-modal transformers, we design a unified transformer architecture, named TransFusion, to fuse cues from both current views and neighboring views. Moreover, we propose the concept of epipolar field to encode 3D positional information into the transformer model. The 3D position encoding guided by the epipolar field provides an efficient way of encoding correspondences between pixels of different views. Experiments on Human 3.6M and Ski-Pose show that our method is more efficient and has consistent improvements compared to other fusion methods. Specifically, we achieve 25.8 mm MPJPE on Human 3.6M with only 5M parameters on 256 x 256 resolution.