# Posture-Informed Muscular Force Learning for Robust Hand Pressure Estimation

arXiv: https://arxiv.org/abs/2410.23629 · HTML: https://mlsanity.com/paper/2410.23629/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 81 references that are in the map.
- First posted on arXiv 31 Oct 2024.

## Builds on (most cited first, 30 of 32)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?](https://mlsanity.com/paper/1711.09577/) (2017)
- [GRAB: A Dataset of Whole-Body Human Grasping of Objects](https://mlsanity.com/paper/2008.11200/) (2020)
- [DexYCB: A Benchmark for Capturing Hand Grasping of Objects](https://mlsanity.com/paper/2104.04631/) (2021)
- [FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape from Single RGB Images](https://mlsanity.com/paper/1909.04349/) (2019)
- [Revisiting Skeleton-based Action Recognition](https://mlsanity.com/paper/2104.13586/) (2021)
- [InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Image](https://mlsanity.com/paper/2008.09309/) (2020)
- [Hand Keypoint Detection in Single Images using Multiview Bootstrapping](https://mlsanity.com/paper/1704.07809/) (2017)
- [ContactPose: A Dataset of Grasps with Object Contact and Hand Pose](https://mlsanity.com/paper/2007.09545/) (2020)
- [DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulation](https://mlsanity.com/paper/2005.14679/) (2020)
- [MediaPipe Hands: On-device Real-time Hand Tracking](https://mlsanity.com/paper/2006.10214/) (2020)
- [OakInk: A Large-scale Knowledge Repository for Understanding Hand-Object Interaction](https://mlsanity.com/paper/2203.15709/) (2022)
- [CPF: Learning a Contact Potential Field to Model the Hand-Object Interaction](https://mlsanity.com/paper/2012.00924/) (2020)
- [Keypoint Transformer: Solving Joint Identification in Challenging Hands and Object Interactions for Accurate 3D Pose Estimation](https://mlsanity.com/paper/2104.14639/) (2021)
- [Weakly Supervised 3D Hand Pose Estimation via Biomechanical Constraints](https://mlsanity.com/paper/2003.09282/) (2020)
- [What's in your hands? 3D Reconstruction of Generic Objects in Hands](https://mlsanity.com/paper/2204.07153/) (2022)
- [ArtiBoost: Boosting Articulated 3D Hand-Object Pose Estimation via Online Exploration and Synthesis](https://mlsanity.com/paper/2109.05488/) (2021)
- [GelSlim: A High-Resolution, Compact, Robust, and Calibrated Tactile-sensing Finger](https://mlsanity.com/paper/1803.00628/) (2018)
- [gSDF: Geometry-Driven Signed Distance Functions for 3D Hand-Object Reconstruction](https://mlsanity.com/paper/2304.11970/) (2023)
- [Estimating 3D Motion and Forces of Person-Object Interactions from Monocular Video](https://mlsanity.com/paper/1904.02683/) (2019)
- [Use the Force, Luke! Learning to Predict Physical Forces by Simulating Effects](https://mlsanity.com/paper/2003.12045/) (2020)
- [ACR: Attention Collaboration-based Regressor for Arbitrary Two-Hand Reconstruction](https://mlsanity.com/paper/2303.05938/) (2023)
- [PressureVision: Estimating Hand Pressure from a Single RGB Image](https://mlsanity.com/paper/2203.10385/) (2022)
- [EgoPressure: A Dataset for Hand Pressure and Pose Estimation in Egocentric Vision](https://mlsanity.com/paper/2409.02224/) (2024)
- [Efficient Annotation and Learning for 3D Hand Pose Estimation: A Survey](https://mlsanity.com/paper/2206.02257/) (2022)
- [HandBooster: Boosting 3D Hand-Mesh Reconstruction by Conditional Synthesis and Sampling of Hand-Object Interactions](https://mlsanity.com/paper/2403.18575/) (2024)
- [PressureVision++: Estimating Fingertip Pressure from Diverse RGB Images](https://mlsanity.com/paper/2301.02310/) (2023)
- [Estimating Fingertip Forces, Torques, and Local Curvatures from Fingernail Images](https://mlsanity.com/paper/1909.05659/) (2019)
- [Force-Aware Interface via Electromyography for Natural VR/AR Interaction](https://mlsanity.com/paper/2210.01225/) (2022)

## Built on by (most cited first, 3 of 3)

- [EgoTactile: Learning Grasp Pressure for Everyday Objects from Egocentric Video](https://mlsanity.com/paper/2606.09243/) (2026)
- [Visuo-Acoustic Hand Pose and Contact Estimation](https://mlsanity.com/paper/2508.00852/) (2025)
- [EMGBlend: Heterogeneity-Aware Self-Supervised Pretraining for Gesture and Force Decoding](https://mlsanity.com/paper/2609.25582/) (2026)

## Abstract (arXiv)

We present PiMForce, a novel framework that enhances hand pressure estimation by leveraging 3D hand posture information to augment forearm surface electromyography (sEMG) signals. Our approach utilizes detailed spatial information from 3D hand poses in conjunction with dynamic muscle activity from sEMG to enable accurate and robust whole-hand pressure measurements under diverse hand-object interactions. We also developed a multimodal data collection system that combines a pressure glove, an sEMG armband, and a markerless finger-tracking module. We created a comprehensive dataset from 21 participants, capturing synchronized data of hand posture, sEMG signals, and exerted hand pressure across various hand postures and hand-object interaction scenarios using our collection system. Our framework enables precise hand pressure estimation in complex and natural interaction scenarios. Our approach substantially mitigates the limitations of traditional sEMG-based or vision-based methods by integrating 3D hand posture information with sEMG signals. Video demos, data, and code are available online.