# FootFormer: Estimating Stability from Visual Input

arXiv: https://arxiv.org/abs/2510.19170 · HTML: https://mlsanity.com/paper/2510.19170/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 59 references that are in the map.
- First posted on arXiv 22 Oct 2025.

## Builds on (most cited first, 25 of 25)

- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [Convolutional Pose Machines](https://mlsanity.com/paper/1602.00134/) (2016)
- [DeepPose: Human Pose Estimation via Deep Neural Networks](https://mlsanity.com/paper/1312.4659/) (2013)
- [Learning Trajectory Dependencies for Human Motion Prediction](https://mlsanity.com/paper/1908.05436/) (2019)
- [RTMPose: Real-Time Multi-Person Pose Estimation based on MMPose](https://mlsanity.com/paper/2303.07399/) (2023)
- [Towards Accurate Multi-person Pose Estimation in the Wild](https://mlsanity.com/paper/1701.01779/) (2017)
- [SimPoE: Simulated Character Control for 3D Human Pose Estimation](https://mlsanity.com/paper/2104.00683/) (2021)
- [Dynamics-Regulated Kinematic Policy for Egocentric Pose Estimation](https://mlsanity.com/paper/2106.05969/) (2021)
- [Contact and Human Dynamics from Monocular Video](https://mlsanity.com/paper/2007.11678/) (2020)
- [Neural Monocular 3D Human Motion Capture with Physical Awareness](https://mlsanity.com/paper/2105.01057/) (2021)
- [Multi-Context Attention for Human Pose Estimation](https://mlsanity.com/paper/1702.07432/) (2017)
- [Flowing ConvNets for Human Pose Estimation in Videos](https://mlsanity.com/paper/1506.02897/) (2015)
- [Estimating 3D Motion and Forces of Person-Object Interactions from Monocular Video](https://mlsanity.com/paper/1904.02683/) (2019)
- [PoseTrack: Joint Multi-Person Pose Estimation and Tracking](https://mlsanity.com/paper/1611.07727/) (2016)
- [Thin-Slicing Network: A Deep Structured Model for Pose Estimation in Videos](https://mlsanity.com/paper/1703.10898/) (2017)
- [Articulated Pose Estimation by a Graphical Model with Image Dependent Pairwise Relations](https://mlsanity.com/paper/1407.3399/) (2014)
- [MMVP: A Multimodal MoCap Dataset with Vision and Pressure Sensors](https://mlsanity.com/paper/2403.17610/) (2024)
- [UnderPressure: Deep Learning for Foot Contact Detection, Ground Reaction Force Estimation and Footskate Cleanup](https://mlsanity.com/paper/2208.04598/) (2022)
- [Weakly-supervised Learning of Human Dynamics](https://mlsanity.com/paper/2007.08969/) (2020)
- [GroundLink: A Dataset Unifying Human Body Movement and Ground Reaction Dynamics](https://mlsanity.com/paper/2310.03930/) (2023)
- [MoDeep: A Deep Learning Framework Using Motion Features for Human Pose Estimation](https://mlsanity.com/paper/1409.7963/) (2014)
- [Image-based Stability Quantification](https://mlsanity.com/paper/2206.11443/) (2022)
- [Deep-Learning Estimation of Weight Distribution Using Joint Kinematics for Lower-Limb Exoskeleton Control](https://mlsanity.com/paper/2402.04180/) (2024)

## Built on by (most cited first, 1 of 1)

- [Gravity-guided Contact Dynamics Estimation from 3D Human Motions](https://mlsanity.com/paper/2606.08133/) (2026)

## Abstract (arXiv)

We propose FootFormer, a cross-modality approach for jointly predicting human motion dynamics directly from visual input. On multiple datasets, FootFormer achieves statistically significantly better or equivalent estimates of foot pressure distributions, foot contact maps, and center of mass (CoM), as compared with existing methods that generate one or two of those measures. Furthermore, FootFormer achieves SOTA performance in estimating stability-predictive components (CoP, CoM, BoS) used in classic kinesiology metrics. Code and data are available at https://github.com/keatonkraiger/Vision-to-Stability.git.