Dexterous Tactile World Model
On the map: World models → Embodied and robot world models
Abstract (arXiv)
World models for manipulation are typically trained from video, yet the events that determine how manipulation unfolds, such as making and releasing contact, are difficult to observe visually and are often easier to sense through touch. We present the Dexterous Tactile World Model (DTWM), a video world model for future-frame prediction of egocentric manipulation from both observed video and tactile signals from a glove worn on each hand. We condition a pretrained video diffusion transformer on each hand's tactile signal through a zero-initialized residual at the corresponding hand location in the video tokens, while a causal mask prevents predicted frames from accessing future information. Compared with a vision-only model matched in architecture, parameters, and training, DTWM reduces the underestimation of hand motion from 23% to 9%, while reducing the perceptual error in the hand region by 7.4% across three training runs per model. The benefit also increases over the prediction horizon, with the improvement in the later predicted chunks being about 4.1x larger than in the first. DTWM also outperforms other visual-tactile world models under the same setting, and training with touch improves future-frame prediction even when no touch is available at inference. Ablations show that the model benefits from both the magnitude and spatial location of force: replacing the tactile signal with binary contact states, either per hand or per location, increases prediction error. The observed course of the force indicates whether the interaction will persist or change.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Embodied and robot world models, one of 11 directions in World models. Builds on 15 of its 56 references that are in the map. First posted on arXiv 28 Sep 2026.
Builds on
15 of 56 references.
Closest ancestors · 4
Same direction, newest first
- 2026FeelWorld: Visuo-Tactile World Model for Hierarchical Contact Prediction and Planning27 Jul
- 2026ViTacWorld: Scaling Visuo-Tactile World Models for Contact-Rich Robot Manipulation24 Jul
- 2026ContactWorld: What Representations Matter for Vision-Tactile Latent World Models in Contact-Rich Manipulation11 Jun
- 2026Visuo-Tactile World Models5 Feb
Other references · 11
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
- 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
- 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,439
- 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
- 2026TouchWorld: A Predictive and Reactive Tactile Foundation Model for Dexterous Manipulationcited by 9
- 2026TouchAnything: A Dataset and Framework for Bimanual Tactile Estimation from Egocentric Videocited by 6
- 2026Artificial Foveated Perception for Mitigating Shortcut Learning in Robotic Foundation Modelscited by 4
- 2026Blind Dexterous Grasping via Real2Sim2Real Tactile Policy Learningcited by 3
- 2026Decoupling Vision, Language, and Action for Efficient Multi-Task Robot Policiescited by 2
- 2025Coffee: Controllable Diffusion Fine-tuningcited by 1
- 2026OpenLongTail: Generative Scaling of Long-Tail Driving Datacited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026DexTacWAM: A Visuo-Tactile World-Action Model for Dexterous Manipulation
- 2026N0-TWAM: Scaling Tactile-Native World-Action Model for Contact-Rich Manipulation
- 2025World Models for Learning Dexterous Hand-Object Interactions from Human Videos
- 2026HiTac-WAM: A Hierarchical Tactile World Action Model for Contact-Rich Robot Manipulation
- 2026VT-WAM: Visual-Tactile World Action Model for Contact-Rich Manipulation
- 2026ME-Dex 1.0: Bringing Heterogeneous Tactile Sensing into World Action Modeling
- 2026DexTouch-WM: Learning Action-Conditioned Tactile World Models from Human Touch for Dexterous Robot Manipulation
- 2026Dream-Tac: A Unified Tactile World Action Model for Contact-Rich Robot Manipulation