# Trajectory Space Factorization for Deep Video-Based 3D Human Pose Estimation

arXiv: https://arxiv.org/abs/1908.08289 · HTML: https://mlsanity.com/paper/1908.08289/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Aug 2019.

## Built on by (most cited first, 30 of 30)

- [3D Human Pose Estimation with Spatial and Temporal Transformers](https://mlsanity.com/paper/2103.10455/) (2021)
- [MixSTE: Seq2seq Mixed Spatio-Temporal Encoder for 3D Human Pose Estimation in Video](https://mlsanity.com/paper/2203.00859/) (2022)
- [MHFormer: Multi-Hypothesis Transformer for 3D Human Pose Estimation](https://mlsanity.com/paper/2111.12707/) (2021)
- [DiffPose: Toward More Reliable 3D Pose Estimation](https://mlsanity.com/paper/2211.16940/) (2022)
- [Exploiting Temporal Contexts with Strided Transformer for 3D Human Pose Estimation](https://mlsanity.com/paper/2103.14304/) (2021)
- [PoseFormerV2: Exploring Frequency Domain for Efficient and Robust 3D Human Pose Estimation](https://mlsanity.com/paper/2303.17472/) (2023)
- [P-STMO: Pre-Trained Spatial Temporal Many-to-One Model for 3D Human Pose Estimation](https://mlsanity.com/paper/2203.07628/) (2022)
- [Learning Skeletal Graph Neural Networks for Hard 3D Pose Estimation](https://mlsanity.com/paper/2108.07181/) (2021)
- [Conditional Directed Graph Convolution for 3D Human Pose Estimation](https://mlsanity.com/paper/2107.07797/) (2021)
- [GLA-GCN: Global-local Adaptive Graph Convolutional Network for 3D Human Pose Estimation from Monocular Video](https://mlsanity.com/paper/2307.05853/) (2023)
- [Uplift and Upsample: Efficient 3D Human Pose Estimation with Uplifting Transformers](https://mlsanity.com/paper/2210.06110/) (2022)
- [HDFormer: High-order Directed Transformer for 3D Human Pose Estimation](https://mlsanity.com/paper/2302.01825/) (2023)
- [Adaptive Multi-view and Temporal Fusing Transformer for 3D Human Pose Estimation](https://mlsanity.com/paper/2110.05092/) (2021)
- [Improving Robustness and Accuracy via Relative Information Encoding in 3D Human Pose Estimation](https://mlsanity.com/paper/2107.13994/) (2021)
- [Learning Dynamical Human-Joint Affinity for 3D Pose Estimation in Videos](https://mlsanity.com/paper/2109.07353/) (2021)
- [Pose Magic: Efficient and Temporally Consistent Human Pose Estimation with a Hybrid Mamba-GCN Network](https://mlsanity.com/paper/2408.02922/) (2024)
- [CrossFormer: Cross Spatio-Temporal Transformer for 3D Human Pose Estimation](https://mlsanity.com/paper/2203.13387/) (2022)
- [HSTFormer: Hierarchical Spatial-Temporal Transformers for 3D Human Pose Estimation](https://mlsanity.com/paper/2301.07322/) (2023)
- [DH-AUG: DH Forward Kinematics Model Driven Augmentation for 3D Human Pose Estimation](https://mlsanity.com/paper/2207.09303/) (2022)
- [IVT: An End-to-End Instance-guided Video Transformer for 3D Pose Estimation](https://mlsanity.com/paper/2208.03431/) (2022)
- [Motion Projection Consistency Based 3D Human Pose Estimation with Virtual Bones from Monocular Videos](https://mlsanity.com/paper/2106.14706/) (2021)
- [MPM: A Unified 2D-3D Human Pose Representation via Masked Pose Modeling](https://mlsanity.com/paper/2306.17201/) (2023)
- [NanoHTNet: Nano Human Topology Network for Efficient 3D Human Pose Estimation](https://mlsanity.com/paper/2501.15763/) (2025)
- [ConvFormer: Parameter Reduction in Transformer Models for 3D Human Pose Estimation by Leveraging Dynamic Multi-Headed Convolutional Attention](https://mlsanity.com/paper/2304.02147/) (2023)
- [Towards Precise 3D Human Pose Estimation with Multi-Perspective Spatial-Temporal Relational Transformers](https://mlsanity.com/paper/2401.16700/) (2024)
- [Multi-hop graph transformer network for 3D human pose estimation](https://mlsanity.com/paper/2405.03055/) (2024)
- [(Fusionformer):Exploiting the Joint Motion Synergy with Fusion Network Based On Transformer for 3D Human Pose Estimation](https://mlsanity.com/paper/2210.04006/) (2022)
- [Capsules as viewpoint learners for human pose estimation](https://mlsanity.com/paper/2302.06194/) (2023)
- [Refined Temporal Pyramidal Compression-and-Amplification Transformer for 3D Human Pose Estimation](https://mlsanity.com/paper/2309.01365/) (2023)
- [Semantic and Articulated Pedestrian Sensing Onboard a Moving Vehicle](https://mlsanity.com/paper/2309.06313/) (2023)

## Abstract (arXiv)

Existing deep learning approaches on 3d human pose estimation for videos are either based on Recurrent or Convolutional Neural Networks (RNNs or CNNs). However, RNN-based frameworks can only tackle sequences with limited frames because sequential models are sensitive to bad frames and tend to drift over long sequences. Although existing CNN-based temporal frameworks attempt to address the sensitivity and drift problems by concurrently processing all input frames in the sequence, the existing state-of-the-art CNN-based framework is limited to 3d pose estimation of a single frame from a sequential input. In this paper, we propose a deep learning-based framework that utilizes matrix factorization for sequential 3d human poses estimation. Our approach processes all input frames concurrently to avoid the sensitivity and drift problems, and yet outputs the 3d pose estimates for every frame in the input sequence. More specifically, the 3d poses in all frames are represented as a motion matrix factorized into a trajectory bases matrix and a trajectory coefficient matrix. The trajectory bases matrix is precomputed from matrix factorization approaches such as Singular Value Decomposition (SVD) or Discrete Cosine Transform (DCT), and the problem of sequential 3d pose estimation is reduced to training a deep network to regress the trajectory coefficient matrix. We demonstrate the effectiveness of our framework on long sequences by achieving state-of-the-art performances on multiple benchmark datasets. Our source code is available at: https://github.com/jiahaoLjh/trajectory-pose-3d.