mlsanityData to 7 Oct 2026

Adaptive Multi-view and Temporal Fusing Transformer for 3D Human Pose Estimation

arXiv 2110.05092 ↗First posted 11 Oct 2021cs.CVcited by 19

Not on the map. Why?

Abstract (arXiv)

This paper proposes a unified framework dubbed Multi-view and Temporal Fusing Transformer (MTF-Transformer) to adaptively handle varying view numbers and video length without camera calibration in 3D Human Pose Estimation (HPE). It consists of Feature Extractor, Multi-view Fusing Transformer (MFT), and Temporal Fusing Transformer (TFT). Feature Extractor estimates 2D pose from each image and fuses the prediction according to the confidence. It provides pose-focused feature embedding and makes subsequent modules computationally lightweight. MFT fuses the features of a varying number of views with a novel Relative-Attention block. It adaptively measures the implicit relative relationship between each pair of views and reconstructs more informative features. TFT aggregates the features of the whole sequence and predicts 3D pose via a transformer. It adaptively deals with the video of arbitrary length and fully unitizes the temporal information. The migration of transformers enables our model to learn spatial geometry better and preserve robustness for varying application scenarios. We report quantitative and qualitative results on the Human3.6M, TotalCapture, and KTH Multiview Football II. Compared with state-of-the-art methods with camera parameters, MTF-Transformer obtains competitive results and generalizes well to dynamic capture with an arbitrary number of unseen views.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 45 of its 72 references that are in the map. First posted on arXiv 11 Oct 2021.

Builds on

45 of 72 references.

References · 45

  1. 2017Attention Is All You Needcited by 59,932
  2. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  3. 2020Point Transformercited by 1,462
  4. 2017End-to-end Recovery of Human Shape and Posecited by 883
  5. 2021Do Vision Transformers See Like Convolutional Neural Networks?cited by 671
  6. 2020End-to-End Human Pose and Mesh Reconstruction with Transformerscited by 417
  7. 2017A simple yet effective baseline for 3d human pose estimationcited by 401
  8. 20183D human pose estimation in video with temporal convolutions and semi-supervised trainingcited by 401
  9. 2021Rethinking Spatial Dimensions of Vision Transformerscited by 361
  10. 2017Cascaded Pyramid Network for Multi-Person Pose Estimationcited by 341
  11. 20213D Human Pose Estimation with Spatial and Temporal Transformerscited by 271
  12. 2017Integral Human Pose Regressioncited by 255
33 more
  1. 2018Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodiescited by 239
  2. 2016Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Posecited by 220
  3. 2019Semantic Graph Convolutional Networks for 3D Human Pose Regressioncited by 204
  4. 2019Learnable Triangulation of Human Posecited by 160
  5. 2021Neural Human Performer: Learning Generalizable Radiance Fields for Human Performance Renderingcited by 158
  6. 2017Compositional Human Pose Regressioncited by 124
  7. 2020Epipolar Transformerscited by 117
  8. 2020SRNet: Improving Generalization in 3D Human Pose Estimation with a Split-and-Recombine Approachcited by 94
  9. 2021Exploiting Temporal Contexts with Strided Transformer for 3D Human Pose Estimationcited by 94
  10. 2019Generating Multiple Hypotheses for 3D Human Pose Estimation with Mixture Density Networkcited by 92
  11. 2019Cross View Fusion for 3D Human Pose Estimationcited by 90
  12. 2020Motion Guided 3D Pose Estimation from Videoscited by 85
  13. 2019Fast and Robust Multi-Person 3D Pose Estimation from Multiple Viewscited by 83
  14. 2020Cascaded deep monocular 3D human pose estimation with evolutionary training datacited by 80
  15. 2019Monocular 3D Human Pose Estimation by Generation and Ordinal Rankingcited by 76
  16. 20163D Human Pose Estimation from a Single Image via Distance Matrix Regressioncited by 73
  17. 2021Learning Skeletal Graph Neural Networks for Hard 3D Pose Estimationcited by 62
  18. 2020Lightweight Multi-View 3D Pose Estimation through Camera-Disentangled Representationcited by 57
  19. 2017Skeleton-Based Action Recognition Using Spatio-Temporal LSTM Network with Trust Gatescited by 54
  20. 2019HEMlets Pose: Learning Part-Centric Heatmap Triplets for Accurate 3D Human Pose Estimationcited by 50
  21. 2021Recent Advances in Monocular 2D and 3D Human Pose Estimation: A Deep Learning Perspectivecited by 49
  22. 2017Harvesting Multiple Views for Marker-less 3D Human Pose Annotationscited by 48
  23. 2019Learning 3D Human Shape and Pose from Dense Body Partscited by 48
  24. 2020AdaFuse: Adaptive Multiview Fusion for Accurate Human Pose Estimation in the Wildcited by 40
  25. 2020Predicting Camera Viewpoint Improves Cross-dataset Generalization for 3D Human Pose Estimationcited by 36
  26. 2019Multi-task Deep Learning for Real-Time 3D Human Pose Estimation and Action Recognitioncited by 35
  27. 2020MotioNet: 3D Human Motion Reconstruction from Monocular Video with Skeleton Consistencycited by 31
  28. 2019Trajectory Space Factorization for Deep Video-Based 3D Human Pose Estimationcited by 30
  29. 2020Fusing Wearable IMUs with Multi-View Images for Human Pose Estimation: A Geometric Approachcited by 27
  30. 2018A generalizable approach for multi-view 3D human pose regressioncited by 26
  31. 2019DeepFuse: An IMU-Aware Network for Real-Time 3D Human Pose Estimation from Multi-View Imagecited by 14
  32. 2020Towards Efficient Scene Understanding via Squeeze Reasoningcited by 10
  33. 2020LiftFormer: 3D Human Pose Estimation using attention modelscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 4; 2025: 7; 2026: 3.

19 papers.

Sort:

  1. 2023Dynamic Inertial Poser (DynaIP): Part-Based Motion Dynamics Learning for Enhanced Human Pose Estimation with Sparse Inertial Sensorscited by 29
  2. 2024Deep learning for 3D human pose estimation and mesh recovery: A surveycited by 11
  3. 2023Delving Deep into Pixel Alignment Feature for Accurate Multi-view Human Mesh Recoverycited by 10
  4. 2023Neuromorphic High-Frequency 3D Dancing Pose Estimation in Dynamic Environmentcited by 8
  5. 2023HaMuCo: Hand Pose Estimation via Multiview Collaborative Self-Supervised Learningcited by 7
  6. 2023Human Pose-based Estimation, Tracking and Action Recognition with Deep Learning: A Surveycited by 6
  7. 2024UPose3D: Uncertainty-Aware 3D Human Pose Estimation with Cross-View and Temporal Cuescited by 6
  8. 2024Human Mesh Recovery from Arbitrary Multi-view Imagescited by 5
  9. 2025H2OT: Hierarchical Hourglass Tokenizer for Efficient Video Pose Transformerscited by 5
  10. 2025Progressive Inertial Poser: Progressive Real-Time Kinematic Chain Estimation for 3D Full-Body Pose from Three IMU Sensorscited by 4
  11. 2025Every Angle Is Worth A Second Glance: Mining Kinematic Skeletal Structures from Multi-view Joint Cloudcited by 2
  12. 2025An uncertainty-aware framework for data-efficient multi-view animal pose estimationcited by 2
7 more
  1. 2026Unconstrained Multi-view Human Pose Estimation with Algebraic Priorscited by 1
  2. 2024Self-learning Canonical Space for Multi-view 3D Human Pose Estimationcited by 0
  3. 2025DeProPose: Deficiency-Proof 3D Human Pose Estimation via Adaptive Multi-View Fusioncited by 0
  4. 2025C-DOG: Multi-View Multi-instance Feature Association Using Connected δ-Overlap Graphscited by 0
  5. 2025Learning What Helps: Task-Aligned Context Selection for Vision Taskscited by 0
  6. 2026HandMvNet: Real-Time 3D Hand Pose Estimation Using Multi-View Cross-Attention Fusioncited by 0
  7. 2026STA-TFM: Spatio-Temporal Aggregation Across Views TransForMer for Pose Estimationcited by 0