mlsanityData to 7 Oct 2026

VTLA: Vision-Tactile-Language-Action Model with Preference Learning for Insertion Manipulation

arXiv 2505.09577 ↗First posted 14 May 2025cs.ROcited by 54

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

While vision-language models have advanced significantly, their application in language-conditioned robotic manipulation is still underexplored, especially for contact-rich tasks that extend beyond visually dominant pick-and-place scenarios. To bridge this gap, we introduce Vision-Tactile-Language-Action model, a novel framework that enables robust policy generation in contact-intensive scenarios by effectively integrating visual and tactile inputs through cross-modal language grounding. A low-cost, multi-modal dataset has been constructed in a simulation environment, containing vision-tactile-action-instruction pairs specifically designed for the fingertip insertion task. Furthermore, we introduce Direct Preference Optimization (DPO) to offer regression-like supervision for the VTLA model, effectively bridging the gap between classification-based next token prediction loss and continuous robotic tasks. Experimental results show that the VTLA model outperforms traditional imitation learning methods (e.g., diffusion policies) and existing multi-modal baselines (TLA/VLA), achieving over 90% success rates on unseen peg shapes. Finally, we conduct real-world peg-in-hole experiments to demonstrate the exceptional Sim2Real performance of the proposed VTLA model. For supplementary videos and results, please visit our project website: https://sites.google.com/view/vtla

Key facts. Cited by 54 papers in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. First posted on arXiv 14 May 2025.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 15; 2026: 39.

54 papers.

Sort:

By area:

  1. 2025ForceVLA: Enhancing VLA Models with a Force-aware MoE for Contact-rich Manipulationcited by 87
  2. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applicationscited by 82
  3. 2025Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Surveycited by 66
  4. 2025Pure Vision Language Action (VLA) Models: A Comprehensive Surveycited by 46
  5. 2025OmniVTLA: Vision-Tactile-Language-Action Models with Semantic-Aligned Tactile Sensingcited by 37
  6. 2026RLDX-1 Technical Reportcited by 25
  7. 2025Parallels Between VLA Model Post-Training and Human Motor Learning: Progress, Challenges, and Trendscited by 17
  8. 2026TacVLA: Contact-Aware Tactile Fusion for Robust Vision-Language-Action Manipulationcited by 17
  9. 2026Dream-Tac: A Unified Tactile World Action Model for Contact-Rich Robot Manipulationcited by 17
  10. 2026VTAM: Video-Tactile-Action Models for Complex Physical Interaction Beyond VLAscited by 14
  11. 2026TaF-VLA: Tactile-Force Alignment in Vision-Language-Action Models for Force-aware Manipulationcited by 11
  12. 2026AnyTouch 2: General Optical Tactile Representation Learning For Dynamic Tactile Perceptioncited by 11
38 more
  1. 2025Is your VLM Sky-Ready? A Comprehensive Spatial Intelligence Benchmark for UAV Navigationcited by 10
  2. 2026T-Rex: Tactile-Reactive Dexterous Manipulationcited by 10
  3. 2026VT-WAM: Visual-Tactile World Action Model for Contact-Rich Manipulationcited by 10
  4. 2025VisualTrans: A Benchmark for Real-World Visual Transformation Reasoningcited by 9
  5. 2025METIS: Multi-Source Egocentric Training for Integrated Dexterous Vision-Language-Action Modelcited by 9
  6. 2026HapticVLA: Contact-Rich Manipulation via Vision-Language-Action Model without Inference-Time Tactile Sensingcited by 9
  7. 2026AT-VLA: Adaptive Tactile Injection for Enhanced Feedback Reaction in Vision-Language-Action Modelscited by 9
  8. 2025End-to-End Dexterous Arm-Hand VLA Policies via Shared Autonomy: VR Teleoperation Augmented by Autonomous Hand VLA Policy for Efficient Data Collectioncited by 8
  9. 2025Survey of Vision-Language-Action Models for Embodied Manipulationcited by 7
  10. 2025An Anatomy of Vision-Language-Action Models: From Modules to Milestones and Challengescited by 4
  11. 2026TORL-VLA: Tactile Guided Online Reinforcement Learning for Contact-Rich Manipulationcited by 4
  12. 2026UniTacVLA: Unified Tactile Understanding and Prediction in Vision Language Action Modelscited by 4
  13. 2026Human-Centric Transferable Tactile Pre-Training for Dexterous Robotic Manipulationcited by 4
  14. 2026DECO: Decoupled Multimodal Diffusion Transformer for Bimanual Dexterous Manipulation with a Plugin Tactile Adaptercited by 3
  15. 2026TacCoRL: Integrating Tactile Feedback into VLA via Simulationcited by 3
  16. 2026UniTac: A Unified Multimodal Model for Cross-Sensor Tactile Understanding and Generationcited by 3
  17. 2026TacWAM: Anchor-Guided World Action Model with Mechanics-Aware Tactile Predictioncited by 3
  18. 2026Tactile Modality Fusion for Vision-Language-Action Modelscited by 2
  19. 2026Modular Sensory Stream for Integrating Physical Feedback in Vision-Language-Action Modelscited by 2
  20. 2026Tactile-based Multimodal Fusion in Embodied Intelligence: A Survey of Vision, Language, and Contact-Driven Paradigmscited by 2
  21. 2026GesVLA: Gesture-Aware Vision-Language-Action Model Embedded Representationscited by 2
  22. 2026Seeing Touch from Motion: A Unified Modality-Aware Visuo-Tactile Policy with Tactile Motion Correlationcited by 2
  23. 2026SoftVTBench: A Safety-Aware Visuo-Tactile Benchmark for Physically Constrained Robotic Manipulation of Deformable Objects (Early Version)cited by 2
  24. 2026FM-VLA: Force-based Memory for Vision-Language-Action Models in Contact-Rich Manipulationcited by 2
  25. 2026τ: Learning Touch-Augmented Vision-Language-Action Models from Future Visual Supervisioncited by 2
  26. 2025Audio-VLA: Adding Contact Audio Perception to Vision-Language-Action Model for Robotic Manipulationcited by 1
  27. 2026Selective Perception for Robot: Task-Aware Attention in Multimodal VLAcited by 1
  28. 2026ThermoAct:Thermal-Aware Vision-Language-Action Models for Robotic Perception and Decision-Makingcited by 1
  29. 2026VL-DPO: Vision-Language-Guided Finetuning for Preference-Aligned Autonomous Drivingcited by 1
  30. 2026TacPAC: Tactile Prediction and Real-Time Action Correction in World-Action Models for Contact-Rich Manipulationcited by 1
  31. 2026World-Action Models for Robot Learning and Control: A Surveycited by 1
  32. 2025Learning to Navigate Socially Through Proactive Risk Perceptioncited by 0
  33. 2025LeCoT: revisiting network architecture for two-view correspondence pruningcited by 0
  34. 2026OmniVaT: Single Domain Generalization for Multimodal Visual-Tactile Learningcited by 0
  35. 2026Symmetry-Aware Fusion of Vision and Tactile Sensing via Bilateral Force Priors for Robotic Manipulationcited by 0
  36. 2026The Lie We Tell: Correcting the Euclidean Fallacy in Vision Language Action Policies via Score Matching on Tangent Spacecited by 0
  37. 2026TAP-VLA: Tactile Annotation Prompting for Vision Language Action Modelscited by 0
  38. 2026What Language Does and What the Evidence Supports: A Functional Role Taxonomy and Evidence Audit of Language Grounding in Embodied Agentscited by 0

50 most cited of 54 All 54 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025TLA: Tactile-Language-Action Model for Contact-Rich Manipulation
  2. 2026N0-VTLA: Scaling Vision-Tactile-Language-Action Model with Latent Tactile Tokens
  3. 2026SimpleTouch: Can Vision-Language-Action Models Master Contact-Rich Manipulation Without Tactile Policy Pretraining?
  4. 2025VLA-Touch: Enhancing Vision-Language-Action Models with Dual-Level Tactile Feedback
  5. 2025Tactile-VLA: Unlocking Vision-Language-Action Model's Physical Knowledge for Tactile Generalization
  6. 2026TacZero: Training-Free Peg Insertion Using a General-Purpose Vision-Language Model with Tactile Feedback
  7. 2026Representation-Aligned Tactile Grounding for Contact-Rich Robotic Manipulation ↗
  8. 2026Tabero: Learning Gentle Manipulation with Closed-Loop Force Feedback from Vision, Touch, and Language