# Sim2Real Manipulation on Unknown Objects with Tactile-based Reinforcement Learning

arXiv: https://arxiv.org/abs/2403.12170 · HTML: https://mlsanity.com/paper/2403.12170/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 37 of its 59 references that are in the map.
- First posted on arXiv 18 Mar 2024.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 30 of 37)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [SAPIEN: A SimulAted Part-based Interactive ENvironment](https://mlsanity.com/paper/2003.08515/) (2020)
- [PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understanding](https://mlsanity.com/paper/1812.02713/) (2018)
- [DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulation](https://mlsanity.com/paper/2005.14679/) (2020)
- [Unsupervised Training for 3D Morphable Model Regression](https://mlsanity.com/paper/1806.06098/) (2018)
- [Rotating without Seeing: Towards In-hand Dexterity through Touch](https://mlsanity.com/paper/2303.10880/) (2023)
- [ObjectFolder: A Dataset of Objects with Implicit Visual, Auditory, and Tactile Representations](https://mlsanity.com/paper/2109.07991/) (2021)
- [See, Hear, and Feel: Smart Sensory Fusion for Robotic Manipulation](https://mlsanity.com/paper/2212.03858/) (2022)
- [TACTO: A Fast, Flexible, and Open-source Simulator for High-Resolution Vision-based Tactile Sensors](https://mlsanity.com/paper/2012.08456/) (2020)
- [Robot Synesthesia: In-Hand Manipulation with Visuotactile Sensing](https://mlsanity.com/paper/2312.01853/) (2023)
- [Tactile-RL for Insertion: Generalization to Objects of Unknown Geometry](https://mlsanity.com/paper/2104.01167/) (2021)
- [Cable Manipulation with a Tactile-Reactive Gripper](https://mlsanity.com/paper/1910.02860/) (2019)
- [GelSlim3.0: High-Resolution Measurement of Shape, Force and Slip in a Compact Tactile-Sensing Finger](https://mlsanity.com/paper/2103.12269/) (2021)
- [Breaking Bad: A Dataset for Geometric Fracture and Reassembly](https://mlsanity.com/paper/2210.11463/) (2022)
- [OmniTact: A Multi-Directional High Resolution Touch Sensor](https://mlsanity.com/paper/2003.06965/) (2020)
- [Sim-to-Real for Robotic Tactile Sensing via Physics-Based Simulation and Learned Latent Projections](https://mlsanity.com/paper/2103.16747/) (2021)
- [Visuotactile Affordances for Cloth Manipulation with Local Control](https://mlsanity.com/paper/2212.05108/) (2022)
- [Tactile Object Pose Estimation from the First Touch with Geometric Contact Rendering](https://mlsanity.com/paper/2012.05205/) (2020)
- [Tactile Sim-to-Real Policy Transfer via Real-to-Sim Image Translation](https://mlsanity.com/paper/2106.08796/) (2021)
- [Dense Tactile Force Distribution Estimation using GelSlim and inverse FEM](https://mlsanity.com/paper/1810.04621/) (2018)
- [Learning to Read Braille: Bridging the Tactile Reality Gap with Diffusion Models](https://mlsanity.com/paper/2304.01182/) (2023)
- [Towards Learning to Play Piano with Dexterous Hands and Touch](https://mlsanity.com/paper/2106.02040/) (2021)
- [Soft-bubble: A highly compliant dense geometry tactile sensor for robot manipulation](https://mlsanity.com/paper/1904.02252/) (2019)
- [Close the Optical Sensing Domain Gap by Physics-Grounded Active Stereo Sensor Simulation](https://mlsanity.com/paper/2201.11924/) (2022)
- [GelTip: A Finger-shaped Optical Tactile Sensor for Robotic Manipulation](https://mlsanity.com/paper/2008.05404/) (2020)
- [Generation of GelSight Tactile Images for Sim2Real Learning](https://mlsanity.com/paper/2101.07169/) (2021)
- [Zero-shot sim-to-real transfer of tactile control policies for aggressive swing-up manipulation](https://mlsanity.com/paper/2101.02680/) (2021)
- [In-Hand Object Pose Tracking via Contact Feedback and GPU-Accelerated Robotic Simulation](https://mlsanity.com/paper/2002.12160/) (2020)
- [Towards Learning to Detect and Predict Contact Events on Vision-based Tactile Sensors](https://mlsanity.com/paper/1910.03973/) (2019)

## Built on by (most cited first, 11 of 11)

- [3D-ViTac: Learning Fine-Grained Manipulation with Visuo-Tactile Sensing](https://mlsanity.com/paper/2410.24091/) (2024)
- [SplatSim: Zero-Shot Sim2Real Transfer of RGB Manipulation Policies Using Gaussian Splatting](https://mlsanity.com/paper/2409.10161/) (2024)
- [Touch in the Wild: Learning Fine-Grained Manipulation with a Portable Visuo-Tactile Gripper](https://mlsanity.com/paper/2507.15062/) (2025)
- [Multi-Modal Manipulation via Multi-Modal Policy Consensus](https://mlsanity.com/paper/2509.23468/) (2025)
- [VT-Refine: Learning Bimanual Assembly with Visuo-Tactile Feedback via Simulation Fine-Tuning](https://mlsanity.com/paper/2510.14930/) (2025)
- [FlexiTac: A Low-Cost, Open-Source, Scalable Tactile Sensing Solution for Robotic Systems](https://mlsanity.com/paper/2604.28156/) (2026)
- [FlashSAC: Fast and Stable Off-Policy Reinforcement Learning for High-Dimensional Robot Control](https://mlsanity.com/paper/2604.04539/) (2026)
- [Data-Efficient Multitask DAgger](https://mlsanity.com/paper/2509.25466/) (2025)
- [Learning Pivoting Manipulation with Force and Vision Feedback Using Optimization-based Demonstrations](https://mlsanity.com/paper/2508.01082/) (2025)
- [Placeit! A Framework for Learning Robot Object Placement Skills](https://mlsanity.com/paper/2510.09267/) (2025)
- [Tac2Pix: Image-Space Visuo-Tactile Fusion for Dexterous Manipulation](https://mlsanity.com/paper/2606.08765/) (2026)

## Abstract (arXiv)

Using tactile sensors for manipulation remains one of the most challenging problems in robotics. At the heart of these challenges is generalization: How can we train a tactile-based policy that can manipulate unseen and diverse objects? In this paper, we propose to perform Reinforcement Learning with only visual tactile sensing inputs on diverse objects in a physical simulator. By training with diverse objects in simulation, it enables the policy to generalize to unseen objects. However, leveraging simulation introduces the Sim2Real transfer problem. To mitigate this problem, we study different tactile representations and evaluate how each affects real-robot manipulation results after transfer. We conduct our experiments on diverse real-world objects and show significant improvements over baselines for the pivoting task. Our project page is available at https://tactilerl.github.io/.