# TacGNN:Learning Tactile-based In-hand Manipulation with a Blind Robot

arXiv: https://arxiv.org/abs/2304.00736 · HTML: https://mlsanity.com/paper/2304.00736/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Apr 2023.

## Built on by (most cited first, 11 of 11)

- [A Survey of Robotic Navigation and Manipulation with Physics Simulators in the Era of Embodied AI](https://mlsanity.com/paper/2505.01458/) (2025)
- [HyperTaxel: Hyper-Resolution for Taxel-Based Tactile Signals Through Contrastive Learning](https://mlsanity.com/paper/2408.08312/) (2024)
- [V-HOP: Visuo-Haptic 6D Object Pose Tracking](https://mlsanity.com/paper/2502.17434/) (2025)
- [Enhancing Tactile-based Reinforcement Learning for Robotic Control](https://mlsanity.com/paper/2510.21609/) (2025)
- [Machine Learning Meets Advanced Robotic Manipulation](https://mlsanity.com/paper/2309.12560/) (2023)
- [Grasp Like Humans: Learning Generalizable Multi-Fingered Grasping from Human Proprioceptive Sensorimotor Integration](https://mlsanity.com/paper/2509.08354/) (2025)
- [roto 2.0: The Robot Tactile Olympiad](https://mlsanity.com/paper/2605.21429/) (2026)
- [The Role of Tactile Sensing for Learning Reach and Grasp](https://mlsanity.com/paper/2502.20367/) (2025)
- [Tac2Motion: Contact-Aware Reinforcement Learning with Tactile Feedback for Robotic Hand Manipulation](https://mlsanity.com/paper/2509.17812/) (2025)
- [Tactile Memory with Soft Robot: Robust Object Insertion via Masked Encoding and Soft Wrist](https://mlsanity.com/paper/2601.19275/) (2026)
- [Tactile-JEPA: Topology-Aware Self-Supervised Representation Learning for Distributed Tactile Sensors](https://mlsanity.com/paper/2609.24385/) (2026)

## Abstract (arXiv)

In this paper, we propose a novel framework for tactile-based dexterous manipulation learning with a blind anthropomorphic robotic hand, i.e. without visual sensing. First, object-related states were extracted from the raw tactile signals by a graph-based perception model - TacGNN. The resulting tactile features were then utilized in the policy learning of an in-hand manipulation task in the second stage. This method was examined by a Baoding ball task - simultaneously manipulating two spheres around each other by 180 degrees in hand. We conducted experiments on object states prediction and in-hand manipulation using a reinforcement learning algorithm (PPO). Results show that TacGNN is effective in predicting object-related states during manipulation by decreasing the RMSE of prediction to 0.096cm comparing to other methods, such as MLP, CNN, and GCN. Finally, the robot hand could finish an in-hand manipulation task solely relying on the robotic own perception - tactile sensing and proprioception. In addition, our methods are tested on three tasks with different difficulty levels and transferred to the real robot without further training.