# Visuo-Tactile World Models

arXiv: https://arxiv.org/abs/2602.06001 · HTML: https://mlsanity.com/paper/2602.06001/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Placed in Embodied and robot world models, one of 11 directions in World models.
- Builds on 14 of its 45 references that are in the map.
- First posted on arXiv 5 Feb 2026.

## Placed in

- World models / [Embodied and robot world models](https://mlsanity.com/wm/embodied-and-robot-world-models/)

## Builds on (most cited first, 14 of 14)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [World Models](https://mlsanity.com/paper/1803.10122/) (2018)
- [Learning Latent Dynamics for Planning from Pixels](https://mlsanity.com/paper/1811.04551/) (2018)
- [Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture](https://mlsanity.com/paper/2301.08243/) (2023)
- [Cosmos World Foundation Model Platform for Physical AI](https://mlsanity.com/paper/2501.03575/) (2025)
- [V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning](https://mlsanity.com/paper/2506.09985/) (2025)
- [Genie: Generative Interactive Environments](https://mlsanity.com/paper/2402.15391/) (2024)
- [GAIA-1: A Generative World Model for Autonomous Driving](https://mlsanity.com/paper/2309.17080/) (2023)
- [Learning Interactive Real-World Simulators](https://mlsanity.com/paper/2310.06114/) (2023)
- [DayDreamer: World Models for Physical Robot Learning](https://mlsanity.com/paper/2206.14176/) (2022)
- [Diffusion for World Modeling: Visual Details Matter in Atari](https://mlsanity.com/paper/2405.12399/) (2024)
- [GAIA-2: A Controllable Multi-View Generative World Model for Autonomous Driving](https://mlsanity.com/paper/2503.20523/) (2025)
- [Learning to Poke by Poking: Experiential Learning of Intuitive Physics](https://mlsanity.com/paper/1606.07419/) (2016)
- [MoDem-V2: Visuo-Motor World Models for Real-World Robot Manipulation](https://mlsanity.com/paper/2309.14236/) (2023)

## Built on by (most cited first, 29 of 29)

- [World Action Models: The Next Frontier in Embodied AI](https://mlsanity.com/paper/2605.12090/) (2026)
- [World Model for Robot Learning: A Comprehensive Survey](https://mlsanity.com/paper/2605.00080/) (2026)
- [Dream-Tac: A Unified Tactile World Action Model for Contact-Rich Robot Manipulation](https://mlsanity.com/paper/2606.08737/) (2026)
- [TacForeSight: Force-Guided Tactile World Model for Contact-Rich Manipulation](https://mlsanity.com/paper/2606.11184/) (2026)
- [VT-WAM: Visual-Tactile World Action Model for Contact-Rich Manipulation](https://mlsanity.com/paper/2607.02503/) (2026)
- [WorldArena 2.0: Extending Embodied World Model Benchmarking on Modality, Functionality and Platform](https://mlsanity.com/paper/2605.17912/) (2026)
- [Tactile-WAM: Touch-Aware World Action Model with Tactile Asymmetric Attention](https://mlsanity.com/paper/2606.26663/) (2026)
- [ViTacWorld: Scaling Visuo-Tactile World Models for Contact-Rich Robot Manipulation](https://mlsanity.com/paper/2607.22530/) (2026)
- [ContactWorld: What Representations Matter for Vision-Tactile Latent World Models in Contact-Rich Manipulation](https://mlsanity.com/paper/2606.13877/) (2026)
- [FeelWorld: Visuo-Tactile World Model for Hierarchical Contact Prediction and Planning](https://mlsanity.com/paper/2607.24267/) (2026)
- [TacWAM: Anchor-Guided World Action Model with Mechanics-Aware Tactile Prediction](https://mlsanity.com/paper/2607.28391/) (2026)
- [Inference-time Policy Steering via Vision and Touch](https://mlsanity.com/paper/2606.14981/) (2026)
- [Imitation from Heterogeneous Demonstrations using Grounded Latent-Action World Models](https://mlsanity.com/paper/2606.21672/) (2026)
- [N0-TWAM: Scaling Tactile-Native World-Action Model for Contact-Rich Manipulation](https://mlsanity.com/paper/2607.23783/) (2026)
- [HiTac-WAM: A Hierarchical Tactile World Action Model for Contact-Rich Robot Manipulation](https://mlsanity.com/paper/2608.19574/) (2026)
- [TacForcing: Streaming Action Generation with Execution-Time Tactile Feedback](https://mlsanity.com/paper/2608.25798/) (2026)
- [Tac-DINO: Learning Vision-Tactile Features with Patch Alignment](https://mlsanity.com/paper/2606.12069/) (2026)
- [TACO: TActile World Model as a Self-COrrector for Scalable Robot Policy Post-Training](https://mlsanity.com/paper/2607.02840/) (2026)
- [DREAMSTEER: Latent World Models Can Steer VLA Policies During Deployment Without Any Finetuning](https://mlsanity.com/paper/2607.02865/) (2026)
- [Learning Physical Interaction: A Survey of Tactile- and Force-aware Robot Learning](https://mlsanity.com/paper/2608.07558/) (2026)
- [Disentangling Visuo-Tactile Foresight: Oracle-Guided Interface Discovery for World Action Models](https://mlsanity.com/paper/2608.00547/) (2026)
- [Compact Visuotactile World Models for Lifting: Prediction, Reward Alignment, and Force Constraints](https://mlsanity.com/paper/2609.09597/) (2026)
- [ProxiDex: Learning Dynamics-Guided Proximity Policy for Dexterous Manipulation](https://mlsanity.com/paper/2609.16586/) (2026)
- [ME-Dex 1.0: Bringing Heterogeneous Tactile Sensing into World Action Modeling](https://mlsanity.com/paper/2609.21449/) (2026)
- [HapticWAM: Distilling Imagined Touch into a World-Action Model without Inference-Time Tactile Sensing](https://mlsanity.com/paper/2609.23888/) (2026)
- [DexTacWAM: A Visuo-Tactile World-Action Model for Dexterous Manipulation](https://mlsanity.com/paper/2609.24976/) (2026)
- [Dexterous Tactile World Model](https://mlsanity.com/paper/2609.34286/) (2026)
- [Temporal Visuo-Tactile Learning for Dexterous Grasp Stability](https://mlsanity.com/paper/2610.10283/) (2026)
- [DexFLEX: Contact-Aware Foundation Controller for Command-Guided Dexterity](https://mlsanity.com/paper/2610.14372/) (2026)

## Abstract (arXiv)

We introduce multi-task Visuo-Tactile World Models (VT-WM), which capture the physics of contact through touch reasoning. By complementing vision with tactile sensing, VT-WM better understands robot-object interactions in contact-rich tasks, avoiding common failure modes of vision-only models under occlusion or ambiguous contact states, such as objects disappearing, teleporting, or moving in ways that violate basic physics. Trained across a set of contact-rich manipulation tasks, VT-WM improves physical fidelity in imagination, achieving 33% better performance at maintaining object permanence and 29% better compliance with the laws of motion in autoregressive rollouts. Moreover, experiments show that grounding in contact dynamics also translates to planning. In zero-shot real-robot experiments, VT-WM achieves up to 35% higher success rates, with the largest gains in multi-step, contact-rich tasks. Finally, VT-WM demonstrates significant downstream versatility, effectively adapting its learned contact dynamics to a novel task and achieving reliable planning success with only a limited set of demonstrations.