# Tac2Real: Reliable and GPU Visuotactile Simulation for Online Reinforcement Learning and Zero-Shot Real-World Deployment

arXiv: https://arxiv.org/abs/2603.28475 · HTML: https://mlsanity.com/paper/2603.28475/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 25 of its 52 references that are in the map.
- First posted on arXiv 30 Mar 2026.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 25 of 25)

- [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://mlsanity.com/paper/2108.10470/) (2021)
- [Ray: A Distributed Framework for Emerging AI Applications](https://mlsanity.com/paper/1712.05889/) (2017)
- [Reactive Diffusion Policy: Slow-Fast Visual-Tactile Policy Learning for Contact-Rich Manipulation](https://mlsanity.com/paper/2503.02881/) (2025)
- [SplatSim: Zero-Shot Sim2Real Transfer of RGB Manipulation Policies Using Gaussian Splatting](https://mlsanity.com/paper/2409.10161/) (2024)
- [TACTO: A Fast, Flexible, and Open-source Simulator for High-Resolution Vision-based Tactile Sensors](https://mlsanity.com/paper/2012.08456/) (2020)
- [An End-to-End Differentiable Framework for Contact-Aware Robot Design](https://mlsanity.com/paper/2107.07501/) (2021)
- [Humanoid-Gym: Reinforcement Learning for Humanoid Robot with Zero-Shot Sim2Real Transfer](https://mlsanity.com/paper/2404.05695/) (2024)
- [Codimensional Incremental Potential Contact](https://mlsanity.com/paper/2012.04457/) (2020)
- [MimicTouch: Leveraging Multi-modal Human Tactile Demonstrations for Contact-rich Manipulation](https://mlsanity.com/paper/2310.16917/) (2023)
- [TacSL: A Library for Visuotactile Sensor Simulation and Learning](https://mlsanity.com/paper/2408.06506/) (2024)
- [DIFFTACTILE: A Physics-based Differentiable Tactile Simulator for Contact-rich Robotic Manipulation](https://mlsanity.com/paper/2403.08716/) (2024)
- [A Modular Robotic Arm Control Stack for Research: Franka-Interface and FrankaPy](https://mlsanity.com/paper/2011.02398/) (2020)
- [Benchmarking the Sim-to-Real Gap in Cloth Manipulation](https://mlsanity.com/paper/2310.09543/) (2023)
- [Analysis of Randomization Effects on Sim2Real Transfer in Reinforcement Learning for Robotic Manipulation Tasks](https://mlsanity.com/paper/2206.06282/) (2022)
- [PolyTouch: A Robust Multi-Modal Tactile Sensor for Contact-rich Manipulation Using Tactile-Diffusion Policies](https://mlsanity.com/paper/2504.19341/) (2025)
- [High-Fidelity Simulated Data Generation for Real-World Zero-Shot Robotic Manipulation Learning with Gaussian Splatting](https://mlsanity.com/paper/2510.10637/) (2025)
- [Tacchi: A Pluggable and Low Computational Cost Elastomer Deformation Simulator for Optical Tactile Sensors](https://mlsanity.com/paper/2301.08343/) (2023)
- [Taccel: Scaling Up Vision-based Tactile Robotics via High-performance GPU Simulation](https://mlsanity.com/paper/2504.12908/) (2025)
- [GIPC: Fast and stable Gauss-Newton optimization of IPC barrier energy](https://mlsanity.com/paper/2308.09400/) (2023)
- [TacIPC: Intersection- and Inversion-free FEM-based Elastomer Simulation For Optical Tactile Sensors](https://mlsanity.com/paper/2311.05843/) (2023)
- [IPC-GraspSim: Reducing the Sim2Real Gap for Parallel-Jaw Grasping with the Incremental Potential Contact Model](https://mlsanity.com/paper/2111.01391/) (2021)
- [Tacchi 2.0: A Low Computational Cost and Comprehensive Dynamic Contact Simulator for Vision-based Tactile Sensors](https://mlsanity.com/paper/2503.09100/) (2025)
- [High-Order Incremental Potential Contact for Elastodynamic Simulation on Curved Meshes](https://mlsanity.com/paper/2205.13727/) (2022)
- [Preconditioned Nonlinear Conjugate Gradient Method for Real-time Interior-point Hyperelasticity](https://mlsanity.com/paper/2405.08001/) (2024)
- [Closing the Reality Gap: Zero-Shot Sim-to-Real Deployment for Dexterous Force-Based Grasping and Manipulation](https://mlsanity.com/paper/2601.02778/) (2026)

## Built on by (most cited first, 1 of 1)

- [Data Pyramid for Embodied Manipulation: A Survey](https://mlsanity.com/paper/2607.24744/) (2026)

## Abstract (arXiv)

Visuotactile sensors are indispensable for contact-rich robotic manipulation tasks. However, policy learning with tactile feedback in simulation, especially for online reinforcement learning (RL), remains a critical challenge, as it demands a delicate balance between physics fidelity and computational efficiency. To address this challenge, we present Tac2Real, a lightweight visuotactile simulation framework designed to enable efficient online RL training. Tac2Real integrates the Preconditioned Nonlinear Conjugate Gradient Incremental Potential Contact (PNCG-IPC) method with a multi-node, multi-GPU high-throughput parallel simulation architecture, which can generate marker displacement fields at interactive rates. Meanwhile, we propose a systematic approach, TacAlign, to narrow both structured and stochastic sources of domain gap, ensuring a reliable zero-shot sim-to-real transfer. We further evaluate Tac2Real on the contact-rich peg insertion task. The zero-shot transfer results achieve a high success rate in the real-world scenario, verifying the effectiveness and robustness of our framework. The project page is: https://ningyurichard.github.io/tac2real-project-page/