# TacEx: GelSight Tactile Simulation in Isaac Sim -- Combining Soft-Body and Visuotactile Simulators

arXiv: https://arxiv.org/abs/2411.04776 · HTML: https://mlsanity.com/paper/2411.04776/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Nov 2024.

## Built on by (most cited first, 12 of 12)

- [Learning to Feel the Future: DreamTacVLA for Contact-Rich Manipulation](https://mlsanity.com/paper/2512.23864/) (2025)
- [A Survey of Robotic Navigation and Manipulation with Physics Simulators in the Era of Embodied AI](https://mlsanity.com/paper/2505.01458/) (2025)
- [OceanSim: A GPU-Accelerated Underwater Robot Perception Simulation Framework](https://mlsanity.com/paper/2503.01074/) (2025)
- [Data Pyramid for Embodied Manipulation: A Survey](https://mlsanity.com/paper/2607.24744/) (2026)
- [HydroShear: Hydroelastic Shear Simulation for Tactile Sim-to-Real Reinforcement Learning](https://mlsanity.com/paper/2603.00446/) (2026)
- [Apple: Toward General Active Perception via Reinforcement Learning](https://mlsanity.com/paper/2505.06182/) (2025)
- [Closing the Reality Gap: Zero-Shot Sim-to-Real Deployment for Dexterous Force-Based Grasping and Manipulation](https://mlsanity.com/paper/2601.02778/) (2026)
- [SoftVTBench: A Safety-Aware Visuo-Tactile Benchmark for Physically Constrained Robotic Manipulation of Deformable Objects (Early Version)](https://mlsanity.com/paper/2607.04234/) (2026)
- [Tac-DINO: Learning Vision-Tactile Features with Patch Alignment](https://mlsanity.com/paper/2606.12069/) (2026)
- [ControlTac: Scaling Tactile Data with Physically Controlled Tactile Image Generation](https://mlsanity.com/paper/2505.20498/) (2025)
- [N0-Foundation: Towards the Age of Tactile Intelligence](https://mlsanity.com/paper/2608.29601/) (2026)
- [A Minimal Optical-Flow Representation for Vision-Based Tactile Rotation Classification in Robotic Manipulation Across Gravity Domains](https://mlsanity.com/paper/2610.12073/) (2026)

## Abstract (arXiv)

Training robot policies in simulation is becoming increasingly popular; nevertheless, a precise, reliable, and easy-to-use tactile simulator for contact-rich manipulation tasks is still missing. To close this gap, we develop TacEx -- a modular tactile simulation framework. We embed a state-of-the-art soft-body simulator for contacts named GIPC and vision-based tactile simulators Taxim and FOTS into Isaac Sim to achieve robust and plausible simulation of the visuotactile sensor GelSight Mini. We implement several Isaac Lab environments for Reinforcement Learning (RL) leveraging our TacEx simulation, including object pushing, lifting, and pole balancing. We validate that the simulation is stable and that the high-dimensional observations, such as the gel deformation and the RGB images from the GelSight camera, can be used for training. The code, videos, and additional results will be released online https://sites.google.com/view/tacex.