mlsanityData to 14 Oct 2026

A tutorial note on collecting simulated data for vision-language-action models

arXiv 2508.06547 ↗First posted 6 Aug 2025cs.ROcited by 0

On the map: Vision-language-action models → Generalist VLA models and benchmarks

Abstract (arXiv)

Traditional robotic systems typically decompose intelligence into independent modules for computer vision, natural language processing, and motion control. Vision-Language-Action (VLA) models fundamentally transform this approach by employing a single neural network that can simultaneously process visual observations, understand human instructions, and directly output robot actions -- all within a unified framework. However, these systems are highly dependent on high-quality training datasets that can capture the complex relationships between visual observations, language instructions, and robotic actions. This tutorial reviews three representative systems: the PyBullet simulation framework for flexible customized data generation, the LIBERO benchmark suite for standardized task definition and evaluation, and the RT-X dataset collection for large-scale multi-robot data acquisition. We demonstrated dataset generation approaches in PyBullet simulation and customized data collection within LIBERO, and provide an overview of the characteristics and roles of the RT-X dataset for large-scale multi-robot data acquisition.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Generalist VLA models and benchmarks, one of 13 directions in Vision-language-action models. Builds on 6 of its 12 references that are in the map. First posted on arXiv 6 Aug 2025.

Builds on

6 of 12 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024OpenVLA: An Open-Source Vision-Language-Action Model13 Jun
  2. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control28 Jul

Other references · 4

  1. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  2. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,410
  3. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 862
  4. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 511

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applications
  2. 2025XR-1: Towards Versatile Vision-Language-Action Models via Learning Unified Vision-Motion Representations
  3. 2026Vision-Language-Action in Robotics: A Survey of Datasets, Benchmarks, and Data Engines
  4. 2025Vision Language Action Models in Robotic Manipulation: A Systematic Review
  5. 2025A Survey on Efficient Vision-Language-Action Models
  6. 2026Vision-Language-Action Models: Experimental Insights from a Real-World UR5 Platform
  7. 2026VLAFlow: A Unified Training Framework for Vision-Language-Action Models via Co-training and Future Latent Alignment
  8. 2024LLaRA: Supercharging Robot Learning Data for Vision-Language Policy