# 3D Pose Nowcasting: Forecast the Future to Improve the Present

arXiv: https://arxiv.org/abs/2308.12914 · HTML: https://mlsanity.com/paper/2308.12914/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 31 of its 63 references that are in the map.
- First posted on arXiv 24 Aug 2023.

## Builds on (most cited first, 30 of 31)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [A simple yet effective baseline for 3d human pose estimation](https://mlsanity.com/paper/1705.03098/) (2017)
- [DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agents](https://mlsanity.com/paper/1704.04394/) (2017)
- [Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Pose](https://mlsanity.com/paper/1611.07828/) (2016)
- [Deep Learning-Based Human Pose Estimation: A Survey](https://mlsanity.com/paper/2012.13392/) (2020)
- [Human Pose Estimation with Iterative Error Feedback](https://mlsanity.com/paper/1507.06550/) (2015)
- [QuaterNet: A Quaternion-based Recurrent Model for Human Motion](https://mlsanity.com/paper/1805.06485/) (2018)
- [V2V-PoseNet: Voxel-to-Voxel Prediction Network for Accurate 3D Hand and Human Pose Estimation from a Single Depth Map](https://mlsanity.com/paper/1711.07399/) (2017)
- [Space-Time-Separable Graph Convolutional Network for Pose Forecasting](https://mlsanity.com/paper/2110.04573/) (2021)
- [Spatial Temporal Transformer Network for Skeleton-based Action Recognition](https://mlsanity.com/paper/2012.06399/) (2020)
- [Action-Agnostic Human Pose Forecasting](https://mlsanity.com/paper/1810.09676/) (2018)
- [A2J: Anchor-to-Joint Regression Network for 3D Articulated Pose Estimation from a Single Depth Image](https://mlsanity.com/paper/1908.09999/) (2019)
- [TRiPOD: Human Trajectory and Pose Dynamics Forecasting in the Wild](https://mlsanity.com/paper/2104.04029/) (2021)
- [Watch It Move: Unsupervised Discovery of 3D Joints for Re-Posing of Articulated Objects](https://mlsanity.com/paper/2112.11347/) (2021)
- [Camera-to-Robot Pose Estimation from a Single Image](https://mlsanity.com/paper/1911.09231/) (2019)
- [mRI: Multi-modal 3D Human Pose Estimation Dataset using mmWave, RGB-D, and Inertial Sensors](https://mlsanity.com/paper/2210.08394/) (2022)
- [Predicting Future Instance Segmentation by Forecasting Convolutional Features](https://mlsanity.com/paper/1803.11496/) (2018)
- [Single-view robot pose and joint angle estimation via render & compare](https://mlsanity.com/paper/2104.09359/) (2021)
- [Motron: Multimodal Probabilistic Human Motion Forecasting](https://mlsanity.com/paper/2203.04132/) (2022)
- [Disentangling Human Dynamics for Pedestrian Locomotion Forecasting with Noisy Supervision](https://mlsanity.com/paper/1911.01138/) (2019)
- [Pose Forecasting in Industrial Human-Robot Collaboration](https://mlsanity.com/paper/2208.07308/) (2022)
- [DECA: Deep viewpoint-Equivariant human pose estimation using Capsule Autoencoders](https://mlsanity.com/paper/2108.08557/) (2021)
- [CRAVES: Controlling Robotic Arm with a Vision-based Economic System](https://mlsanity.com/paper/1812.00725/) (2018)
- [Forecasting Human Dynamics from Static Images](https://mlsanity.com/paper/1704.03432/) (2017)
- [Kinect Range Sensing: Structured-Light versus Time-of-Flight Kinect](https://mlsanity.com/paper/1505.05459/) (2015)
- [Human Pose Forecasting via Deep Markov Models](https://mlsanity.com/paper/1707.09240/) (2017)
- [Forecasting Characteristic 3D Poses of Human Actions](https://mlsanity.com/paper/2011.15079/) (2020)
- [Towards Good Practices for Deep 3D Hand Pose Estimation](https://mlsanity.com/paper/1707.07248/) (2017)
- [NTU-X: An Enhanced Large-scale Dataset for Improving Pose-based Recognition of Subtle Human Actions](https://mlsanity.com/paper/2101.11529/) (2021)

## Abstract (arXiv)

Technologies to enable safe and effective collaboration and coexistence between humans and robots have gained significant importance in the last few years. A critical component useful for realizing this collaborative paradigm is the understanding of human and robot 3D poses using non-invasive systems. Therefore, in this paper, we propose a novel vision-based system leveraging depth data to accurately establish the 3D locations of skeleton joints. Specifically, we introduce the concept of Pose Nowcasting, denoting the capability of the proposed system to enhance its current pose estimation accuracy by jointly learning to forecast future poses. The experimental evaluation is conducted on two different datasets, providing accurate and real-time performance and confirming the validity of the proposed method on both the robotic and human scenarios.