# Development and Evaluation of a Learning-based Model for Real-time Haptic Texture Rendering

arXiv: https://arxiv.org/abs/2212.13332 · HTML: https://mlsanity.com/paper/2212.13332/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 40 references that are in the map.
- First posted on arXiv 27 Dec 2022.

## Builds on (most cited first, 7 of 7)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Tacotron: Towards End-to-End Speech Synthesis](https://mlsanity.com/paper/1703.10135/) (2017)
- [Interactive Perception: Leveraging Action in Perception and Perception in Action](https://mlsanity.com/paper/1604.03670/) (2016)
- [Four Years in Review: Statistical Practices of Likert Scales in Human-Robot Interaction Studies](https://mlsanity.com/paper/2001.03231/) (2020)
- [Vibrotactile Signal Generation from Texture Images or Attributes using Generative Adversarial Network](https://mlsanity.com/paper/1902.07480/) (2019)
- [Learning an Action-Conditional Model for Haptic Texture Generation](https://mlsanity.com/paper/1909.13025/) (2019)

## Built on by (most cited first, 1 of 1)

- [Why Modeling Human Haptic Material Perception with AI Is Difficult](https://mlsanity.com/paper/2605.16602/) (2026)

## Abstract (arXiv)

Current Virtual Reality (VR) environments lack the rich haptic signals that humans experience during real-life interactions, such as the sensation of texture during lateral movement on a surface. Adding realistic haptic textures to VR environments requires a model that generalizes to variations of a user's interaction and to the wide variety of existing textures in the world. Current methodologies for haptic texture rendering exist, but they usually develop one model per texture, resulting in low scalability. We present a deep learning-based action-conditional model for haptic texture rendering and evaluate its perceptual performance in rendering realistic texture vibrations through a multi part human user study. This model is unified over all materials and uses data from a vision-based tactile sensor (GelSight) to render the appropriate surface conditioned on the user's action in real time. For rendering texture, we use a high-bandwidth vibrotactile transducer attached to a 3D Systems Touch device. The result of our user study shows that our learning-based method creates high-frequency texture renderings with comparable or better quality than state-of-the-art methods without the need for learning a separate model per texture. Furthermore, we show that the method is capable of rendering previously unseen textures using a single GelSight image of their surface.