# ACROSS: A Deformation-Based Cross-Modal Representation for Robotic Tactile Perception

arXiv: https://arxiv.org/abs/2411.08533 · HTML: https://mlsanity.com/paper/2411.08533/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 30 references that are in the map.
- First posted on arXiv 13 Nov 2024.

## Builds on (most cited first, 16 of 16)

- [Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering](https://mlsanity.com/paper/1606.09375/) (2016)
- [Generating 3D faces using Convolutional Mesh Autoencoders](https://mlsanity.com/paper/1807.10267/) (2018)
- [DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulation](https://mlsanity.com/paper/2005.14679/) (2020)
- [TACTO: A Fast, Flexible, and Open-source Simulator for High-Resolution Vision-based Tactile Sensors](https://mlsanity.com/paper/2012.08456/) (2020)
- [Taxim: An Example-based Simulation Model for GelSight Tactile Sensors](https://mlsanity.com/paper/2109.04027/) (2021)
- [GelSlim3.0: High-Resolution Measurement of Shape, Force and Slip in a Compact Tactile-Sensing Finger](https://mlsanity.com/paper/2103.12269/) (2021)
- [Sim-to-Real for Robotic Tactile Sensing via Physics-Based Simulation and Learned Latent Projections](https://mlsanity.com/paper/2103.16747/) (2021)
- [ViTac: Feature Sharing between Vision and Tactile Sensing for Cloth Texture Recognition](https://mlsanity.com/paper/1802.07490/) (2018)
- [TactileGCN: A Graph Convolutional Network for Predicting Grasp Stability with Tactile Sensors](https://mlsanity.com/paper/1901.06181/) (2019)
- ["Touching to See" and "Seeing to Feel": Robotic Cross-modal SensoryData Generation for Visual-Tactile Perception](https://mlsanity.com/paper/1902.06273/) (2019)
- [Visuo-Haptic Object Perception for Robots: An Overview](https://mlsanity.com/paper/2203.11544/) (2022)
- [Interpreting and Predicting Tactile Signals for the SynTouch BioTac](https://mlsanity.com/paper/2101.05452/) (2021)
- [Learning to Synthesize Volumetric Meshes from Vision-based Tactile Imprints](https://mlsanity.com/paper/2203.15155/) (2022)
- [Optimizing BioTac Simulation for Realistic Tactile Perception](https://mlsanity.com/paper/2404.10425/) (2024)
- [Transferring Tactile Data Across Sensors](https://mlsanity.com/paper/2410.14310/) (2024)
- [A Biomimetic Fingerprint for Robotic Tactile Sensing](https://mlsanity.com/paper/2307.00937/) (2023)

## Built on by (most cited first, 2 of 2)

- [Advances in Compliance Detection: Novel Models Using Vision-Based Tactile Sensors](https://mlsanity.com/paper/2506.14980/) (2025)
- [SPLIT: Separating Physical-Contact via Latent Arithmetic in Image-Based Tactile Sensors](https://mlsanity.com/paper/2604.24449/) (2026)

## Abstract (arXiv)

Tactile perception is essential for human interaction with the environment and is becoming increasingly crucial in robotics. Tactile sensors like the BioTac mimic human fingertips and provide detailed interaction data. Despite its utility in applications like slip detection and object identification, this sensor is now deprecated, making many valuable datasets obsolete. However, recreating similar datasets with newer sensor technologies is both tedious and time-consuming. Therefore, adapting these existing datasets for use with new setups and modalities is crucial. In response, we introduce ACROSS, a novel framework for translating data between tactile sensors by exploiting sensor deformation information. We demonstrate the approach by translating BioTac signals into the DIGIT sensor. Our framework consists of first converting the input signals into 3D deformation meshes. We then transition from the 3D deformation mesh of one sensor to the mesh of another, and finally convert the generated 3D deformation mesh into the corresponding output space. We demonstrate our approach to the most challenging problem of going from a low-dimensional tactile representation to a high-dimensional one. In particular, we transfer the tactile signals of a BioTac sensor to DIGIT tactile images. Our approach enables the continued use of valuable datasets and data exchange between groups with different setups.