# Advances in Compliance Detection: Novel Models Using Vision-Based Tactile Sensors

arXiv: https://arxiv.org/abs/2506.14980 · HTML: https://mlsanity.com/paper/2506.14980/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 42 references that are in the map.
- First posted on arXiv 17 Jun 2025.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet](https://mlsanity.com/paper/2101.11986/) (2021)
- [An Introduction to Convolutional Neural Networks](https://mlsanity.com/paper/1511.08458/) (2015)
- [Long-term Recurrent Convolutional Networks for Visual Recognition and Description](https://mlsanity.com/paper/1411.4389/) (2014)
- [Orbit: A Unified Simulation Framework for Interactive Robot Learning Environments](https://mlsanity.com/paper/2301.04195/) (2023)
- [A Touch, Vision, and Language Dataset for Multimodal Alignment](https://mlsanity.com/paper/2402.13232/) (2024)
- [Shape-independent Hardness Estimation Using Deep Learning and a GelSight Tactile Sensor](https://mlsanity.com/paper/1704.03955/) (2017)
- [Visuo-Haptic Object Perception for Robots: An Overview](https://mlsanity.com/paper/2203.11544/) (2022)
- [DefGraspSim: Physics-based simulation of grasp outcomes for 3D deformable objects](https://mlsanity.com/paper/2203.11274/) (2022)
- [Low-Cost Teleoperation with Haptic Feedback through Vision-based Tactile Sensors for Rigid and Soft Object Manipulation](https://mlsanity.com/paper/2403.16764/) (2024)
- [Transferring Tactile Data Across Sensors](https://mlsanity.com/paper/2410.14310/) (2024)
- [ACROSS: A Deformation-Based Cross-Modal Representation for Robotic Tactile Perception](https://mlsanity.com/paper/2411.08533/) (2024)
- [Learning Object Compliance via Young's Modulus from Single Grasps using Camera-Based Tactile Sensors](https://mlsanity.com/paper/2406.15304/) (2024)

## Abstract (arXiv)

Compliance is a critical parameter for describing objects in engineering, agriculture, and biomedical applications. Traditional compliance detection methods are limited by their lack of portability and scalability, rely on specialized, often expensive equipment, and are unsuitable for robotic applications. Moreover, existing neural network-based approaches using vision-based tactile sensors still suffer from insufficient prediction accuracy. In this paper, we propose two models based on Long-term Recurrent Convolutional Networks (LRCNs) and Transformer architectures that leverage RGB tactile images and other information captured by the vision-based sensor GelSight to predict compliance metrics accurately. We validate the performance of these models using multiple metrics and demonstrate their effectiveness in accurately estimating compliance. The proposed models exhibit significant performance improvement over the baseline. Additionally, we investigated the correlation between sensor compliance and object compliance estimation, which revealed that objects that are harder than the sensor are more challenging to estimate.