# Adaptive Multi-view and Temporal Fusing Transformer for 3D Human Pose Estimation

arXiv: https://arxiv.org/abs/2110.05092 · HTML: https://mlsanity.com/paper/2110.05092/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 72 references that are in the map.
- First posted on arXiv 11 Oct 2021.

## Builds on (most cited first, 30 of 45)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Point Transformer](https://mlsanity.com/paper/2011.00931/) (2020)
- [End-to-end Recovery of Human Shape and Pose](https://mlsanity.com/paper/1712.06584/) (2017)
- [Do Vision Transformers See Like Convolutional Neural Networks?](https://mlsanity.com/paper/2108.08810/) (2021)
- [End-to-End Human Pose and Mesh Reconstruction with Transformers](https://mlsanity.com/paper/2012.09760/) (2020)
- [3D human pose estimation in video with temporal convolutions and semi-supervised training](https://mlsanity.com/paper/1811.11742/) (2018)
- [A simple yet effective baseline for 3d human pose estimation](https://mlsanity.com/paper/1705.03098/) (2017)
- [Rethinking Spatial Dimensions of Vision Transformers](https://mlsanity.com/paper/2103.16302/) (2021)
- [Cascaded Pyramid Network for Multi-Person Pose Estimation](https://mlsanity.com/paper/1711.07319/) (2017)
- [3D Human Pose Estimation with Spatial and Temporal Transformers](https://mlsanity.com/paper/2103.10455/) (2021)
- [Integral Human Pose Regression](https://mlsanity.com/paper/1711.08229/) (2017)
- [Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodies](https://mlsanity.com/paper/1801.01615/) (2018)
- [Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Pose](https://mlsanity.com/paper/1611.07828/) (2016)
- [Semantic Graph Convolutional Networks for 3D Human Pose Regression](https://mlsanity.com/paper/1904.03345/) (2019)
- [Learnable Triangulation of Human Pose](https://mlsanity.com/paper/1905.05754/) (2019)
- [Neural Human Performer: Learning Generalizable Radiance Fields for Human Performance Rendering](https://mlsanity.com/paper/2109.07448/) (2021)
- [Compositional Human Pose Regression](https://mlsanity.com/paper/1704.00159/) (2017)
- [Epipolar Transformers](https://mlsanity.com/paper/2005.04551/) (2020)
- [SRNet: Improving Generalization in 3D Human Pose Estimation with a Split-and-Recombine Approach](https://mlsanity.com/paper/2007.09389/) (2020)
- [Exploiting Temporal Contexts with Strided Transformer for 3D Human Pose Estimation](https://mlsanity.com/paper/2103.14304/) (2021)
- [Generating Multiple Hypotheses for 3D Human Pose Estimation with Mixture Density Network](https://mlsanity.com/paper/1904.05547/) (2019)
- [Cross View Fusion for 3D Human Pose Estimation](https://mlsanity.com/paper/1909.01203/) (2019)
- [Motion Guided 3D Pose Estimation from Videos](https://mlsanity.com/paper/2004.13985/) (2020)
- [Fast and Robust Multi-Person 3D Pose Estimation from Multiple Views](https://mlsanity.com/paper/1901.04111/) (2019)
- [Cascaded deep monocular 3D human pose estimation with evolutionary training data](https://mlsanity.com/paper/2006.07778/) (2020)
- [Monocular 3D Human Pose Estimation by Generation and Ordinal Ranking](https://mlsanity.com/paper/1904.01324/) (2019)
- [3D Human Pose Estimation from a Single Image via Distance Matrix Regression](https://mlsanity.com/paper/1611.09010/) (2016)
- [Learning Skeletal Graph Neural Networks for Hard 3D Pose Estimation](https://mlsanity.com/paper/2108.07181/) (2021)
- [Lightweight Multi-View 3D Pose Estimation through Camera-Disentangled Representation](https://mlsanity.com/paper/2004.02186/) (2020)

## Built on by (most cited first, 19 of 19)

- [Dynamic Inertial Poser (DynaIP): Part-Based Motion Dynamics Learning for Enhanced Human Pose Estimation with Sparse Inertial Sensors](https://mlsanity.com/paper/2312.02196/) (2023)
- [Deep learning for 3D human pose estimation and mesh recovery: A survey](https://mlsanity.com/paper/2402.18844/) (2024)
- [Delving Deep into Pixel Alignment Feature for Accurate Multi-view Human Mesh Recovery](https://mlsanity.com/paper/2301.06020/) (2023)
- [Neuromorphic High-Frequency 3D Dancing Pose Estimation in Dynamic Environment](https://mlsanity.com/paper/2301.06648/) (2023)
- [HaMuCo: Hand Pose Estimation via Multiview Collaborative Self-Supervised Learning](https://mlsanity.com/paper/2302.00988/) (2023)
- [Human Pose-based Estimation, Tracking and Action Recognition with Deep Learning: A Survey](https://mlsanity.com/paper/2310.13039/) (2023)
- [Human Mesh Recovery from Arbitrary Multi-view Images](https://mlsanity.com/paper/2403.12434/) (2024)
- [UPose3D: Uncertainty-Aware 3D Human Pose Estimation with Cross-View and Temporal Cues](https://mlsanity.com/paper/2404.14634/) (2024)
- [H2OT: Hierarchical Hourglass Tokenizer for Efficient Video Pose Transformers](https://mlsanity.com/paper/2509.06956/) (2025)
- [Progressive Inertial Poser: Progressive Real-Time Kinematic Chain Estimation for 3D Full-Body Pose from Three IMU Sensors](https://mlsanity.com/paper/2505.05336/) (2025)
- [Every Angle Is Worth A Second Glance: Mining Kinematic Skeletal Structures from Multi-view Joint Cloud](https://mlsanity.com/paper/2502.02936/) (2025)
- [An uncertainty-aware framework for data-efficient multi-view animal pose estimation](https://mlsanity.com/paper/2510.09903/) (2025)
- [Unconstrained Multi-view Human Pose Estimation with Algebraic Priors](https://mlsanity.com/paper/2604.24312/) (2026)
- [Self-learning Canonical Space for Multi-view 3D Human Pose Estimation](https://mlsanity.com/paper/2403.12440/) (2024)
- [DeProPose: Deficiency-Proof 3D Human Pose Estimation via Adaptive Multi-View Fusion](https://mlsanity.com/paper/2502.16419/) (2025)
- [C-DOG: Multi-View Multi-instance Feature Association Using Connected δ-Overlap Graphs](https://mlsanity.com/paper/2507.14095/) (2025)
- [Learning What Helps: Task-Aligned Context Selection for Vision Tasks](https://mlsanity.com/paper/2512.00489/) (2025)
- [HandMvNet: Real-Time 3D Hand Pose Estimation Using Multi-View Cross-Attention Fusion](https://mlsanity.com/paper/2608.20093/) (2026)
- [STA-TFM: Spatio-Temporal Aggregation Across Views TransForMer for Pose Estimation](https://mlsanity.com/paper/2609.24482/) (2026)

## Abstract (arXiv)

This paper proposes a unified framework dubbed Multi-view and Temporal Fusing Transformer (MTF-Transformer) to adaptively handle varying view numbers and video length without camera calibration in 3D Human Pose Estimation (HPE). It consists of Feature Extractor, Multi-view Fusing Transformer (MFT), and Temporal Fusing Transformer (TFT). Feature Extractor estimates 2D pose from each image and fuses the prediction according to the confidence. It provides pose-focused feature embedding and makes subsequent modules computationally lightweight. MFT fuses the features of a varying number of views with a novel Relative-Attention block. It adaptively measures the implicit relative relationship between each pair of views and reconstructs more informative features. TFT aggregates the features of the whole sequence and predicts 3D pose via a transformer. It adaptively deals with the video of arbitrary length and fully unitizes the temporal information. The migration of transformers enables our model to learn spatial geometry better and preserve robustness for varying application scenarios. We report quantitative and qualitative results on the Human3.6M, TotalCapture, and KTH Multiview Football II. Compared with state-of-the-art methods with camera parameters, MTF-Transformer obtains competitive results and generalizes well to dynamic capture with an arbitrary number of unseen views.