# Grasp Like Humans: Learning Generalizable Multi-Fingered Grasping from Human Proprioceptive Sensorimotor Integration

arXiv: https://arxiv.org/abs/2509.08354 · HTML: https://mlsanity.com/paper/2509.08354/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 57 references that are in the map.
- First posted on arXiv 10 Sep 2025.

## Builds on (most cited first, 23 of 23)

- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware](https://mlsanity.com/paper/2304.13705/) (2023)
- [Learning Dexterous In-Hand Manipulation](https://mlsanity.com/paper/1808.00177/) (2018)
- [Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots](https://mlsanity.com/paper/2402.10329/) (2024)
- [Causal Confusion in Imitation Learning](https://mlsanity.com/paper/1905.11979/) (2019)
- [MimicPlay: Long-Horizon Imitation Learning by Watching Human Play](https://mlsanity.com/paper/2302.12422/) (2023)
- [Segment and Track Anything](https://mlsanity.com/paper/2305.06558/) (2023)
- [UniDexGrasp: Universal Robotic Dexterous Grasping via Learning Diverse Proposal Generation and Goal-Conditioned Policy](https://mlsanity.com/paper/2303.00938/) (2023)
- [UniDexGrasp++: Improving Dexterous Grasping Policy Learning via Geometry-aware Curriculum and Iterative Generalist-Specialist Learning](https://mlsanity.com/paper/2304.00464/) (2023)
- [See, Hear, and Feel: Smart Sensory Fusion for Robotic Manipulation](https://mlsanity.com/paper/2212.03858/) (2022)
- [Learning Object-Action Relations from Bimanual Human Demonstration Using Graph Networks](https://mlsanity.com/paper/1908.08391/) (2019)
- [SwingBot: Learning Physical Features from In-hand Tactile Exploration for Dynamic Swing-up Manipulation](https://mlsanity.com/paper/2101.11812/) (2021)
- [Learning to Grasp Without Seeing](https://mlsanity.com/paper/1805.04201/) (2018)
- [TactileGCN: A Graph Convolutional Network for Predicting Grasp Stability with Tactile Sensors](https://mlsanity.com/paper/1901.06181/) (2019)
- [FingerSLAM: Closed-loop Unknown Object Localization and Reconstruction from Visuo-tactile Feedback](https://mlsanity.com/paper/2303.07997/) (2023)
- [Relaxed-Rigidity Constraints: Kinematic Trajectory Optimization and Collision Avoidance for In-Grasp Manipulation](https://mlsanity.com/paper/1806.00942/) (2018)
- [Sim-to-Real Transfer for Robotic Manipulation with Tactile Sensory](https://mlsanity.com/paper/2103.00410/) (2021)
- [TacGNN:Learning Tactile-based In-hand Manipulation with a Blind Robot](https://mlsanity.com/paper/2304.00736/) (2023)
- [Multi-Fingered In-Hand Manipulation with Various Object Properties Using Graph Convolutional Networks and Distributed Tactile Sensors](https://mlsanity.com/paper/2205.04169/) (2022)
- [A Wearable Robotic Hand for Hand-over-Hand Imitation Learning](https://mlsanity.com/paper/2309.14860/) (2023)
- [EasyCalib: Simple and Low-Cost In-Situ Calibration for Force Reconstruction with Vision-Based Tactile Sensors](https://mlsanity.com/paper/2403.10256/) (2024)
- [Towards integrated tactile sensorimotor control in anthropomorphic soft robotic hands](https://mlsanity.com/paper/2102.03318/) (2021)
- [TeleFMG: A Wearable Force-Myography Device for Natural Teleoperation of Multi-finger Robotic Hands](https://mlsanity.com/paper/2308.13929/) (2023)

## Built on by (most cited first, 1 of 1)

- [BLaDA: Bridging Language to Functional Dexterous Actions within 3DGS Fields](https://mlsanity.com/paper/2604.08410/) (2026)

## Abstract (arXiv)

Tactile and kinesthetic perceptions are crucial for human dexterous manipulation, enabling reliable grasping of objects via proprioceptive sensorimotor integration. For robotic hands, even though acquiring such tactile and kinesthetic feedback is feasible, establishing a direct mapping from this sensory feedback to motor actions remains challenging. In this paper, we propose a novel glove-mediated tactile-kinematic perception-prediction framework for grasp skill transfer from human intuitive and natural operation to robotic execution based on imitation learning, and its effectiveness is validated through generalized grasping tasks, including those involving deformable objects. Firstly, we integrate a data glove to capture tactile and kinesthetic data at the joint level. The glove is adaptable for both human and robotic hands, allowing data collection from natural human hand demonstrations across different scenarios. It ensures consistency in the raw data format, enabling evaluation of grasping for both human and robotic hands. Secondly, we establish a unified representation of multi-modal inputs based on graph structures with polar coordinates. We explicitly integrate the morphological differences into the designed representation, enhancing the compatibility across different demonstrators and robotic hands. Furthermore, we introduce the Tactile-Kinesthetic Spatio-Temporal Graph Networks (TK-STGN), which leverage multidimensional subgraph convolutions and attention-based LSTM layers to extract spatio-temporal features from graph inputs to predict node-based states for each hand joint. These predictions are then mapped to final commands through a force-position hybrid mapping.