Disentangling Visuo-Tactile Foresight: Oracle-Guided Interface Discovery for World Action Models
On the map: World models → Embodied and robot world models
Abstract (arXiv)
Contact-rich manipulation remains challenging because successful control depends on physical interaction cues that are often weakly observable from vision alone. Recent tactile world action models jointly model future visual observations and tactile signals to guide action generation, but how such futures should be structured for effective use by the action expert remains underexplored. Directly studying this question with learned world action models is difficult because end-to-end behavior entangles physically invalid visual futures, unreliable predictions, inaccurate or cross-modally inconsistent tactile forecasts, and an unreadable future-to-action interface. To make this interface independently studyable, we introduce Oracle Visuo-Tactile Foresight (OVTF), a controlled framework that supplies paired RGB and tactile futures from successful trajectories verified in simulation. By fixing the future provider, OVTF isolates the interface and asks a cleaner question: if the future is successful and physically executable, what representation allows the action expert to absorb its benefit? Within OVTF, we propose Asymmetric Phase-Local Future Memory (AFM), in which visual memory reads future vision, each tactile memory jointly attends to its own tactile stream and phase-aligned future vision, and cross-tactile access is blocked. We compare AFM with Modality-Isolated Future Memory (IFM), which removes visual-to-tactile access and processes each future modality independently. Across seven tasks on the UniVTAC simulation benchmark, AFM achieves 32.0% average success, compared with 23.7% for IFM and 14.9% for UniVTAC-ACT. This controlled comparison shows that selective phase-aligned visual-tactile routing provides a more actionable future-to-action bridge than complete modality isolation.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Embodied and robot world models, one of 11 directions in World models. Builds on 15 of its 21 references that are in the map. First posted on arXiv 1 Aug 2026.
Builds on
15 of 21 references.
Closest ancestors · 11
Same direction, newest first
- 2026KAM-WM: Kinematic Affordance Maps from Latent World Models for Robot Manipulation6 Jul
- 2026VT-WAM: Visual-Tactile World Action Model for Contact-Rich Manipulation2 Jul
- 2026Tactile-WAM: Touch-Aware World Action Model with Tactile Asymmetric Attention25 Jun
- 2026ImageWAM: Do World Action Models Really Need Video Generation, or Just Image Editing?17 Jun
- 2026TacForeSight: Force-Guided Tactile World Model for Contact-Rich Manipulation9 Jun
- 2026Dream-Tac: A Unified Tactile World Action Model for Contact-Rich Robot Manipulation7 Jun
- 2026OA-WAM: Object-Addressable World Action Model for Robust Robot Manipulation7 May
- 2026OmniVTA: Visuo-Tactile World Modeling for Contact-Rich Robotic Manipulation19 Mar
- 2026Fast-WAM: Do World Action Models Need Test-time Future Imagination?17 Mar
- 2026World Action Models are Zero-shot Policies17 Feb
- 2026Visuo-Tactile World Models5 Feb
Other references · 4
- 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
- 2025ViTacFormer: Learning Cross-Modal Representation for Visuo-Tactile Dexterous Manipulationcited by 35
- 2026UniVTAC: A Unified Simulation Platform for Visuo-Tactile Manipulation Data Generation, Learning, and Benchmarkingcited by 15
- 2025Learning to Feel the Future: DreamTacVLA for Contact-Rich Manipulationcited by 14
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026FeelWorld: Visuo-Tactile World Model for Hierarchical Contact Prediction and Planning
- 2026TacDyn-WAM: Learning Implicit Tactile Dynamics in a Heterogeneous Visuo-Tactile World Action Model
- 2026TacPAC: Tactile Prediction and Real-Time Action Correction in World-Action Models for Contact-Rich Manipulation
- 2026HiTac-WAM: A Hierarchical Tactile World Action Model for Contact-Rich Robot Manipulation
- 2026Representation-Aligned Tactile Grounding for Contact-Rich Robotic Manipulation ↗
- 2026TacWAM: Anchor-Guided World Action Model with Mechanics-Aware Tactile Prediction
- 2026Agile-WAM: An Agile Tactile World Action Model for Contact-Rich Robot Control
- 2026τ: Learning Touch-Augmented Vision-Language-Action Models from Future Visual Supervision