# Human Pose Estimation from Ambiguous Pressure Recordings with Spatio-temporal Masked Transformers

arXiv: https://arxiv.org/abs/2303.05691 · HTML: https://mlsanity.com/paper/2303.05691/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 28 references that are in the map.
- First posted on arXiv 10 Mar 2023.

## Builds on (most cited first, 21 of 21)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [A Closer Look at Spatiotemporal Convolutions for Action Recognition](https://mlsanity.com/paper/1711.11248/) (2017)
- [On the Convergence of Adam and Beyond](https://mlsanity.com/paper/1904.09237/) (2019)
- [Simple Baselines for Human Pose Estimation and Tracking](https://mlsanity.com/paper/1804.06208/) (2018)
- [ViTPose: Simple Vision Transformer Baselines for Human Pose Estimation](https://mlsanity.com/paper/2204.12484/) (2022)
- [Revisiting Skeleton-based Action Recognition](https://mlsanity.com/paper/2104.13586/) (2021)
- [Learnable Triangulation of Human Pose](https://mlsanity.com/paper/1905.05754/) (2019)
- [Exploiting Temporal Contexts with Strided Transformer for 3D Human Pose Estimation](https://mlsanity.com/paper/2103.14304/) (2021)
- [P-STMO: Pre-Trained Spatial Temporal Many-to-One Model for 3D Human Pose Estimation](https://mlsanity.com/paper/2203.07628/) (2022)
- [Connecting Touch and Vision via Cross-Modal Prediction](https://mlsanity.com/paper/1906.06322/) (2019)
- [Bodies at Rest: 3D Human Pose and Shape Estimation from a Pressure Image using Synthetic Data](https://mlsanity.com/paper/2004.01166/) (2020)
- [Uncertainty-Aware Adaptation for Self-Supervised 3D Human Pose Estimation](https://mlsanity.com/paper/2203.15293/) (2022)
- [Simultaneously-Collected Multimodal Lying Pose Dataset: Towards In-Bed Human Pose Monitoring under Adverse Vision Conditions](https://mlsanity.com/paper/2008.08735/) (2020)
- [Estimating Pose from Pressure Data for Smart Beds with Deep Image-based Pose Estimators](https://mlsanity.com/paper/2206.06518/) (2022)
- [Identity and Posture Recognition in Smart Beds with Deep Multitask Learning](https://mlsanity.com/paper/2104.02159/) (2021)
- [Pressure Eye: In-bed Contact Pressure Estimation via Contact-less Imaging](https://mlsanity.com/paper/2201.11828/) (2022)

## Built on by (most cited first, 3 of 3)

- [PI-HMR: Towards Robust In-bed Temporal Human Shape Reconstruction with Contact Pressure Sensing](https://mlsanity.com/paper/2503.00068/) (2025)
- [SkelFormer: Markerless 3D Pose and Shape Estimation using Skeletal Transformers](https://mlsanity.com/paper/2404.12625/) (2024)
- [P2P-Insole: Human Pose Estimation Using Foot Pressure Distribution and Motion Sensors](https://mlsanity.com/paper/2505.00755/) (2025)

## Abstract (arXiv)

Despite the impressive performance of vision-based pose estimators, they generally fail to perform well under adverse vision conditions and often don't satisfy the privacy demands of customers. As a result, researchers have begun to study tactile sensing systems as an alternative. However, these systems suffer from noisy and ambiguous recordings. To tackle this problem, we propose a novel solution for pose estimation from ambiguous pressure data. Our method comprises a spatio-temporal vision transformer with an encoder-decoder architecture. Detailed experiments on two popular public datasets reveal that our model outperforms existing solutions in the area. Moreover, we observe that increasing the number of temporal crops in the early stages of the network positively impacts the performance while pre-training the network in a self-supervised setting using a masked auto-encoder approach also further improves the results.