# Semi-Supervised Disentanglement of Tactile Contact~Geometry from Sliding-Induced Shear

arXiv: https://arxiv.org/abs/2208.12500 · HTML: https://mlsanity.com/paper/2208.12500/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 27 references that are in the map.
- First posted on arXiv 26 Aug 2022.

## Builds on (most cited first, 20 of 20)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Context Encoders: Feature Learning by Inpainting](https://mlsanity.com/paper/1604.07379/) (2016)
- [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1603.09246/) (2016)
- [InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets](https://mlsanity.com/paper/1606.03657/) (2016)
- [Disentangling by Factorising](https://mlsanity.com/paper/1802.05983/) (2018)
- [Audio-Visual Scene Analysis with Self-Supervised Multisensory Features](https://mlsanity.com/paper/1804.03641/) (2018)
- [Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Prediction](https://mlsanity.com/paper/1611.09842/) (2016)
- [Swapping Autoencoder for Deep Image Manipulation](https://mlsanity.com/paper/2007.00653/) (2020)
- [Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Tasks](https://mlsanity.com/paper/1810.10191/) (2018)
- [Deep Learning of Representations: Looking Forward](https://mlsanity.com/paper/1305.0445/) (2013)
- [Learning Disentangled Representations with Semi-Supervised Deep Generative Models](https://mlsanity.com/paper/1706.00400/) (2017)
- [Tactile Mapping and Localization from High-Resolution Tactile Imprints](https://mlsanity.com/paper/1904.10944/) (2019)
- [Tactile SLAM: Real-time inference of shape and pose from planar pushing](https://mlsanity.com/paper/2011.07044/) (2020)
- [Soft Biomimetic Optical Tactile Sensing with the TacTip: A Review](https://mlsanity.com/paper/2105.14455/) (2021)
- [Learning rich touch representations through cross-modal self-supervision](https://mlsanity.com/paper/2101.08616/) (2021)
- [From pixels to percepts: Highly robust edge perception and contour following using deep learning and an optical biomimetic tactile sensor](https://mlsanity.com/paper/1812.02941/) (2018)
- [Pose-Based Tactile Servoing: Controlled Soft Touch using Deep Learning](https://mlsanity.com/paper/2012.02504/) (2020)
- [Optimal Deep Learning for Robot Touch](https://mlsanity.com/paper/2003.01916/) (2020)
- [Voronoi Features for Tactile Sensing: Direct Inference of Pressure, Shear, and Contact Locations](https://mlsanity.com/paper/1805.07868/) (2018)
- [Tactile Image-to-Image Disentanglement of Contact Geometry from Motion-Induced Shear](https://mlsanity.com/paper/2109.03615/) (2021)

## Abstract (arXiv)

The sense of touch is fundamental to human dexterity. When mimicked in robotic touch, particularly by use of soft optical tactile sensors, it suffers from distortion due to motion-dependent shear. This complicates tactile tasks like shape reconstruction and exploration that require information about contact geometry. In this work, we pursue a semi-supervised approach to remove shear while preserving contact-only information. We validate our approach by showing a match between the model-generated unsheared images with their counterparts from vertically tapping onto the object. The model-generated unsheared images give faithful reconstruction of contact-geometry otherwise masked by shear, along with robust estimation of object pose then used for sliding exploration and full reconstruction of several planar shapes. We show that our semi-supervised approach achieves comparable performance to its fully supervised counterpart across all validation tasks with an order of magnitude less supervision. The semi-supervised method is thus more computational and labeled sample-efficient. We expect it will have broad applicability to wide range of complex tactile exploration and manipulation tasks performed via a shear-sensitive sense of touch.