RotVLA: Rotational Latent Action for Vision-Language-Action Model
On the map: Vision-language-action models → Human-video and latent-action pretraining
also in 1 more
Abstract (arXiv)
Latent Action Models (LAMs) have emerged as an effective paradigm for handling heterogeneous datasets during Vision-Language-Action (VLA) model pretraining, offering a unified action space across embodiments. However, existing LAMs often rely on discrete quantization encode and decode pipelines, which can lead to trivial frame reconstruction behavior, limited representational capacity, and a lack of physically meaningful structure. We introduce RotVLA, a VLA framework built on a continuous rotational latent action representation. Latent actions are modeled as elements of SO(n), providing continuity, compositionality, and structured geometry aligned with real-world action dynamics. A triplet frame learning framework further enforces meaningful temporal dynamics while avoiding degeneration. RotVLA consists of a VLM backbone and a flow-matching action head, pretrained on large-scale cross-embodiment robotic datasets and human videos with latent-action supervision. For downstream robot control, the flow-matching head is extended into a unified action expert that jointly denoises latent and robot actions. Here, latent actions serve as a latent planner, providing high-level guidance that conditions action generation. With only 1.7B parameters and 1700+ hours of pretraining data, RotVLA achieves 98.2% on LIBERO and 89.6% / 88.5% on RoboTwin2.0 under clean and randomized settings, respectively. It also demonstrates strong real-world performance on manipulation tasks, consistently outperforming existing VLA models.
Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Human-video and latent-action pretraining, one of 13 directions in Vision-language-action models. Builds on 65 of its 74 references that are in the map. First posted on arXiv 13 May 2026.
Builds on
65 of 74 references.
Closest ancestors · 6
Same direction, newest first
- 2026LARY: A Latent Action Representation Yielding Benchmark for Generalizable Vision-to-Action Alignment13 Apr
- 2026UniLACT: Depth-Aware RGB Latent Action Learning for Vision-Language-Action Models23 Feb
- 2026CLAP: Contrastive Latent Action Pretraining for Learning Vision-Language-Action Models from Human Videos7 Jan
- 2025LatBot: Distilling Universal Latent Actions for Vision-Language-Action Models28 Nov
- 2025UniVLA: Learning to Act Anywhere with Task-centric Latent Actions9 May
- 2024Latent Action Pretraining from Videos15 Oct
Other references · 59
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2023Visual Instruction Tuningcited by 7,792
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 2022Scalable Diffusion Models with Transformerscited by 5,931
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2017Neural Discrete Representation Learningcited by 4,650
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
- 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
- 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
- 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
- 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
38 more
- 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
- 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,186
- 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,167
- 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 849
- 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Successcited by 844
- 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
- 2024Genie: Generative Interactive Environmentscited by 706
- 2024RDT-1B: a Diffusion Foundation Model for Bimanual Manipulationcited by 670
- 2025FAST: Efficient Action Tokenization for Vision-Language-Action Modelscited by 643
- 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 634
- 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 517
- 2025RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulationcited by 477
- 2025AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systemscited by 390
- 2025X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Modelcited by 285
- 2025π*0.6: a VLA That Learns From Experiencecited by 280
- 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 274
- 2025Motus: A Unified Latent Action World Modelcited by 231
- 2018RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitationcited by 202
- 2020Spatial-Temporal Transformer Networks for Traffic Flow Forecastingcited by 179
- 2024RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulationcited by 166
- 2023Structured World Models from Human Videoscited by 144
- 2025VLA-Adapter: An Effective Paradigm for Tiny-Scale Vision-Language-Action Modelcited by 140
- 2022VIOLA: Imitation Learning for Vision-Based Manipulation with Object Proposal Priorscited by 135
- 2023FurnitureBench: Reproducible Real-World Benchmark for Long-Horizon Complex Manipulationcited by 132
- 2022From Play to Policy: Conditional Behavior Generation from Uncurated Robot Datacited by 103
- 2026StarVLA: A Lego-like Codebase for Vision-Language-Action Model Developingcited by 103
- 2022Grounding Language with Visual Affordances over Unstructured Datacited by 101
- 2022Robot Learning on the Job: Human-in-the-Loop Autonomy and Learning During Deploymentcited by 101
- 2023On Bringing Robots Homecited by 101
- 2026A Pragmatic VLA Foundation Modelcited by 97
- 2021Bottom-Up Skill Discovery from Unsegmented Demonstrations for Long-Horizon Robot Manipulationcited by 82
- 2024FMB: a Functional Manipulation Benchmark for Generalizable Robotic Learningcited by 79
- 2022Learning and Retrieval from Prior Data for Skill-based Imitation Learningcited by 77
- 2025BridgeVLA: Input-Output Alignment for Efficient 3D Manipulation Learning with Vision-Language Modelscited by 77
- 2025RoboCOIN: An Open-Sourced Bimanual Robotic Data Collection for Integrated Manipulationcited by 72
- 2023MUTEX: Learning Unified Policies from Multimodal Task Specificationscited by 70
- 2024IGOR: Image-GOal Representations are the Atomic Control Units for Foundation Models in Embodied AIcited by 68
- 2023Multi-Stage Cable Routing through Hierarchical Imitation Learningcited by 59
50 most cited of 59 All 59 (JSON)
Built on by
Citing papers per year: 2026: 7.
7 papers.
- 2026EvoScene-VLA: Evolving Scene Beliefs Inside the Action Decoder for Chunked Robot Controlcited by 2
- 2026LIRA: Local Cross-Layer Information Routing for Vision-Language-Action Decodingcited by 2
- 2026Action QFormer: Structured Representation Shaping under Action Supervision in Vision-Language-Action Modelscited by 1
- 2026DLAM: Distributional Latent Actions with Temporal Constraintscited by 1
- 2026WLA3: World Latent Action Modeling for Semantics, Dynamics, and Kinematicscited by 0
- 2026Algebraic Consistency Alone Does Not Certify Temporal Structure in Latent Action Modelscited by 0
- 2026UniWAM: Unified World-Action Modelcited by 0
- 2026UniWAM: Unified World-Action Model1 Oct 2026
- 2026Algebraic Consistency Alone Does Not Certify Temporal Structure in Latent Action Models20 Sep 2026
- 2026WLA3: World Latent Action Modeling for Semantics, Dynamics, and Kinematics14 Sep 2026
- 2026LIRA: Local Cross-Layer Information Routing for Vision-Language-Action Decoding6 Aug 2026
- 2026DLAM: Distributional Latent Actions with Temporal Constraints29 Jul 2026
- 2026Action QFormer: Structured Representation Shaping under Action Supervision in Vision-Language-Action Models16 Jul 2026
- 2026EvoScene-VLA: Evolving Scene Beliefs Inside the Action Decoder for Chunked Robot Control21 May 2026
Similar, not cited
Close in wording, no citation either way
- 2026LARA: Latent Action Representation Alignment for Vision-Language-Action Models
- 2026ViDAL: A Visual Dynamics-Grounded Action Latent Space for Vision-Language-Action Models
- 2026Latent Cluster Analysis for Vision-Language-Action Models
- 2025XR-1: Towards Versatile Vision-Language-Action Models via Learning Unified Vision-Motion Representations
- 2026Joint-Aligned Latent Action: Towards Scalable VLA Pretraining in the Wild
- 2026ALAM: Algebraically Consistent Latent Action Model for Vision-Language-Action Models
- 2026From Pixels to Tokens: A Systematic Study of Latent Action Supervision for Vision-Language-Action Models
- 2026QuoVLA: Quotient Space for Vision-Language-Action Models