# A Modularized Design Approach for GelSight Family of Vision-based Tactile Sensors

arXiv: https://arxiv.org/abs/2504.14739 · HTML: https://mlsanity.com/paper/2504.14739/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 63 references that are in the map.
- First posted on arXiv 20 Apr 2025.

## Builds on (most cited first, 25 of 25)

- [The CMA Evolution Strategy: A Tutorial](https://mlsanity.com/paper/1604.00772/) (2016)
- [DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulation](https://mlsanity.com/paper/2005.14679/) (2020)
- [More Than a Feeling: Learning to Grasp and Regrasp using Vision and Touch](https://mlsanity.com/paper/1805.11085/) (2018)
- [An End-to-End Differentiable Framework for Contact-Aware Robot Design](https://mlsanity.com/paper/2107.07501/) (2021)
- [GelSlim: A High-Resolution, Compact, Robust, and Calibrated Tactile-sensing Finger](https://mlsanity.com/paper/1803.00628/) (2018)
- [Taxim: An Example-based Simulation Model for GelSight Tactile Sensors](https://mlsanity.com/paper/2109.04027/) (2021)
- [GelSlim3.0: High-Resolution Measurement of Shape, Force and Slip in a Compact Tactile-Sensing Finger](https://mlsanity.com/paper/2103.12269/) (2021)
- [GelSight Wedge: Measuring High-Resolution 3D Contact Geometry with a Compact Robot Finger](https://mlsanity.com/paper/2106.08851/) (2021)
- [ShapeMap 3-D: Efficient shape mapping through dense touch and vision](https://mlsanity.com/paper/2109.09884/) (2021)
- [Improved GelSight Tactile Sensor for Measuring Geometry and Slip](https://mlsanity.com/paper/1708.00922/) (2017)
- [9DTact: A Compact Vision-Based Tactile Sensor for Accurate 3D Shape Reconstruction and Generalizable 6D Force Estimation](https://mlsanity.com/paper/2308.14277/) (2023)
- [Dr.Jit: A Just-In-Time Compiler for Differentiable Rendering](https://mlsanity.com/paper/2202.01284/) (2022)
- [Simulation of Vision-based Tactile Sensors using Physics based Rendering](https://mlsanity.com/paper/2012.13184/) (2020)
- [DenseTact: Optical Tactile Sensor for Dense Shape Reconstruction](https://mlsanity.com/paper/2201.01367/) (2022)
- [GelSight Svelte: A Human Finger-shaped Single-camera Tactile Robot Finger with Large Sensing Coverage and Proprioceptive Sensing](https://mlsanity.com/paper/2309.10885/) (2023)
- [GelSight Fin Ray: Incorporating Tactile Sensing into a Soft Compliant Robotic Gripper](https://mlsanity.com/paper/2204.07146/) (2022)
- [Soft, Round, High Resolution Tactile Fingertip Sensors for Dexterous Robotic Manipulation](https://mlsanity.com/paper/2005.09068/) (2020)
- [Digger Finger: GelSight Tactile Sensor for Object Identification Inside Granular Media](https://mlsanity.com/paper/2102.10230/) (2021)
- [Grasp Stability Prediction with Sim-to-Real Transfer from Tactile Sensing](https://mlsanity.com/paper/2208.02885/) (2022)
- [GelSight360: An Omnidirectional Camera-Based Tactile Sensor for Dexterous Robotic Manipulation](https://mlsanity.com/paper/2304.04268/) (2023)
- [Minsight: A Fingertip-Sized Vision-Based Tactile Sensor for Robotic Manipulation](https://mlsanity.com/paper/2304.10990/) (2023)
- [DenseTact-Mini: An Optical Tactile Sensor for Grasping Multi-Scale Objects From Flat Surfaces](https://mlsanity.com/paper/2309.08860/) (2023)
- [GelSight Baby Fin Ray: A Compact, Compliant, Flexible Finger with High-Resolution Tactile Sensing](https://mlsanity.com/paper/2303.14883/) (2023)
- [GelSight EndoFlex: A Soft Endoskeleton Hand with Continuous High-Resolution Tactile Sensing](https://mlsanity.com/paper/2303.17935/) (2023)
- [GelBelt: A Vision-based Tactile Sensor for Continuous Sensing of Large Surfaces](https://mlsanity.com/paper/2501.06263/) (2025)

## Abstract (arXiv)

GelSight family of vision-based tactile sensors has proven to be effective for multiple robot perception and manipulation tasks. These sensors are based on an internal optical system and an embedded camera to capture the deformation of the soft sensor surface, inferring the high-resolution geometry of the objects in contact. However, customizing the sensors for different robot hands requires a tedious trial-and-error process to re-design the optical system. In this paper, we formulate the GelSight sensor design process as a systematic and objective-driven design problem and perform the design optimization with a physically accurate optical simulation. The method is based on modularizing and parameterizing the sensor's optical components and designing four generalizable objective functions to evaluate the sensor. We implement the method with an interactive and easy-to-use toolbox called OptiSense Studio. With the toolbox, non-sensor experts can quickly optimize their sensor design in both forward and inverse ways following our predefined modules and steps. We demonstrate our system with four different GelSight sensors by quickly optimizing their initial design in simulation and transferring it to the real sensors.