Self-Attentive 3D Human Pose and Shape Estimation from Videos
Not on the map. Why?
Abstract (arXiv)
We consider the task of estimating 3D human pose and shape from videos. While existing frame-based approaches have made significant progress, these methods are independently applied to each image, thereby often leading to inconsistent predictions. In this work, we present a video-based learning algorithm for 3D human pose and shape estimation. The key insights of our method are two-fold. First, to address the inconsistent temporal prediction issue, we exploit temporal information in videos and propose a self-attention module that jointly considers short-range and long-range dependencies across frames, resulting in temporally coherent estimations. Second, we model human motion with a forecasting module that allows the transition between adjacent frames to be smooth. We evaluate our method on the 3DPW, MPI-INF-3DHP, and Human3.6M datasets. Extensive experimental results show that our algorithm performs favorably against the state-of-the-art methods.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 55 of its 66 references that are in the map. First posted on arXiv 26 Mar 2021.
Builds on
55 of 66 references.
References · 55
- 2017Attention Is All You Needcited by 59,932
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2017The Kinetics Human Action Video Datasetcited by 1,894
- 2020YOLOv4: Optimal Speed and Accuracy of Object Detectioncited by 1,624
- 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,208
- 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,161
- 2017End-to-end Recovery of Human Shape and Posecited by 883
- 2016Simple Online and Realtime Trackingcited by 862
- 2018Self-Attention Generative Adversarial Networkscited by 809
- 2018Multimodal Unsupervised Image-to-Image Translationcited by 795
38 more
- 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 778
- 2018Image Transformercited by 770
- 2016Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Imagecited by 692
- 2019Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loopcited by 625
- 2019VIBE: Video Inference for Human Body Pose and Shape Estimationcited by 615
- 2016Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervisioncited by 493
- 2019YOLACT: Real-time Instance Segmentationcited by 441
- 2018Learning 3D Human Dynamics from Videocited by 315
- 2018Diverse Image-to-Image Translation via Disentangled Representationscited by 296
- 2019Convolutional Mesh Regression for Single-Image Human Shape Reconstructioncited by 270
- 2016Unsupervised Learning for Physical Interaction through Video Predictioncited by 256
- 2015Structural-RNN: Deep Learning on Spatio-Temporal Graphscited by 234
- 2018CrowdPose: Efficient Crowded Scenes Pose Estimation and A New Benchmarkcited by 214
- 2017Unite the People: Closing the Loop Between 3D and 2D Human Representationscited by 205
- 2017UnFlow: Unsupervised Learning of Optical Flow with a Bidirectional Census Losscited by 202
- 2018Neural Body Fitting: Unifying Deep Learning and Model-Based Human Pose and Shape Estimationcited by 185
- 2017PoseTrack: A Benchmark for Human Pose Estimation and Trackingcited by 173
- 2019Depth from Videos in the Wild: Unsupervised Monocular Depth Learning from Unknown Camerascited by 170
- 2020Monocular Expressive Body Regression through Body-Driven Attentioncited by 155
- 2018DF-Net: Unsupervised Joint Learning of Depth and Flow using Cross-Task Consistencycited by 148
- 2017Unsupervised Learning of Disentangled Representations from Videocited by 138
- 2017VNect: Real-time 3D Human Pose Estimation with a Single RGB Cameracited by 129
- 2015DeepCut: Joint Subset Partition and Labeling for Multi Person Pose Estimationcited by 117
- 2019Exploiting temporal context for 3D human pose estimation in the wildcited by 110
- 2020CrDoCo: Pixel-level Domain Transfer with Cross-Domain Consistencycited by 107
- 2019Self-Supervised Learning of 3D Human Pose using Multi-view Geometrycited by 98
- 2017Deep representation learning for human motion prediction and classificationcited by 97
- 2019Unsupervised 3D Pose Estimation with Geometric Self-Supervisioncited by 95
- 2017Exploiting temporal information for 3D pose estimationcited by 89
- 2017The Pose Knows: Video Forecasting by Generating Pose Futurescited by 88
- 2019Human Mesh Recovery from Monocular Images via a Skeleton-disentangled Representationcited by 87
- 2016An Uncertain Future: Forecasting from Static Images using Variational Autoencoderscited by 86
- 20183D Human Pose Estimation in the Wild by Adversarial Learningcited by 86
- 2018Neural Kinematic Networks for Unsupervised Motion Retargettingcited by 85
- 2019RepNet: Weakly Supervised Training of an Adversarial Reprojection Network for 3D Human Pose Estimationcited by 85
- 2020Synthetic Training for Accurate 3D Human Pose and Shape Estimation in the Wildcited by 82
- 2019Sim2real transfer learning for 3D human pose estimation: motion to the rescuecited by 75
- 2018Mo2Cap2: Real-time Mobile 3D Motion Capture with a Cap-mounted Fisheye Cameracited by 67
50 most cited of 55 All 55 (JSON)
Built on by
Citing papers per year: 2022: 1.
1 paper.