# MAT: Multi-Fingered Adaptive Tactile Grasping via Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/1909.04787 · HTML: https://mlsanity.com/paper/1909.04787/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Sep 2019.

## Built on by (most cited first, 12 of 12)

- [Reset-Free Reinforcement Learning via Multi-Task Learning: Learning Dexterous Manipulation Behaviors without Human Intervention](https://mlsanity.com/paper/2104.11203/) (2021)
- [Factory: Fast Contact for Robotic Assembly](https://mlsanity.com/paper/2205.03532/) (2022)
- [The Power of the Senses: Generalizable Manipulation from Vision and Touch through Masked Multimodal Learning](https://mlsanity.com/paper/2311.00924/) (2023)
- [REBOOT: Reuse Data for Bootstrapping Efficient Real-World Dexterous Manipulation](https://mlsanity.com/paper/2309.03322/) (2023)
- [Improved Learning of Robot Manipulation Tasks via Tactile Intrinsic Motivation](https://mlsanity.com/paper/2102.11051/) (2021)
- [Action Conditioned Tactile Prediction: case study on slip prediction](https://mlsanity.com/paper/2205.09430/) (2022)
- [The Role of Tactile Sensing in Learning and Deploying Grasp Refinement Algorithms](https://mlsanity.com/paper/2109.11234/) (2021)
- [Planning Visual-Tactile Precision Grasps via Complementary Use of Vision and Touch](https://mlsanity.com/paper/2212.08604/) (2022)
- [TacForcing: Streaming Action Generation with Execution-Time Tactile Feedback](https://mlsanity.com/paper/2608.25798/) (2026)
- [Towards Transferring Tactile-based Continuous Force Control Policies from Simulation to Robot](https://mlsanity.com/paper/2311.07245/) (2023)
- [TIAGo RL: Simulated Reinforcement Learning Environments with Tactile Data for Mobile Robots](https://mlsanity.com/paper/2311.07260/) (2023)
- [The Role of Tactile Sensing for Learning Reach and Grasp](https://mlsanity.com/paper/2502.20367/) (2025)

## Abstract (arXiv)

Vision-based grasping systems typically adopt an open-loop execution of a planned grasp. This policy can fail due to many reasons, including ubiquitous calibration error. Recovery from a failed grasp is further complicated by visual occlusion, as the hand is usually occluding the vision sensor as it attempts another open-loop regrasp. This work presents MAT, a tactile closed-loop method capable of realizing grasps provided by a coarse initial positioning of the hand above an object. Our algorithm is a deep reinforcement learning (RL) policy optimized through the clipped surrogate objective within a maximum entropy RL framework to balance exploitation and exploration. The method utilizes tactile and proprioceptive information to act through both fine finger motions and larger regrasp movements to execute stable grasps. A novel curriculum of action motion magnitude makes learning more tractable and helps turn common failure cases into successes. Careful selection of features that exhibit small sim-to-real gaps enables this tactile grasping policy, trained purely in simulation, to transfer well to real world environments without the need for additional learning. Experimentally, this methodology improves over a vision-only grasp success rate substantially on a multi-fingered robot hand. When this methodology is used to realize grasps from coarse initial positions provided by a vision-only planner, the system is made dramatically more robust to calibration errors in the camera-robot transform.