mlsanityData to 7 Oct 2026

CrossVLA: Cross-Paradigm Post-Training and Inference Optimization for Vision-Language-Action Models

arXiv 2605.21854 ↗First posted 21 May 2026cs.CV cs.AIcited by 0

On the map: Vision-language-action models → VLA architectures and generalization

also in 2 more
Abstract (arXiv)

Vision-Language-Action (VLA) models have rapidly converged on a small set of architectural patterns: discrete-token autoregression (e.g. OpenVLA) and continuous-action flow-matching (e.g. pi-0.5). Yet preference alignment via Direct Preference Optimisation (DPO) -- the de-facto post-training step in language models -- has been studied almost exclusively on autoregressive VLAs. We present CrossVLA, an empirical study of cross-paradigm VLA post-training. Three contributions: (i) a surrogate flow-matching log-probability estimator that lets DPO operate on continuous-action backbones without probability-flow ODE integration; (ii) a head-to-head comparison of LoRA and DoRA as the parameter-efficient layer for VLA DPO, finding DoRA improves over OpenVLA SFT by a mean +10.4 pp across LIBERO 4-suite (600 trials, 3 seeds) -- per-suite +20.0 Object, +11.0 Long-horizon, +8.0 Goal, +2.7 Spatial -- with zero seed variance on Object (38/50 on each of 3 seeds); (iii) an inference-time anatomy showing the denoise loop dominates 78.6% of sample_actions latency and prefix-K/V caching a la VLA-Cache caps at a 21% acceleration ceiling -- both chunk-level and token-level cache strategies degrade success rate to 0-80% in our benchmarks. We further pretrain a multi-view + temporal projection head on 6000 LIBERO frames, achieving 99.5% k-NN recall@1 for same-task retrieval (36x over random), available as a downstream initialisation. All code, ckpts, training logs, and reproduction scripts are open at https://github.com/lz-googlefycy/vla-lab.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in VLA architectures and generalization, one of 13 directions in Vision-language-action models. Builds on 43 of its 49 references that are in the map. First posted on arXiv 21 May 2026.

Builds on

43 of 49 references.

References · 43

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  3. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  4. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  5. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  6. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  7. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  8. 2023Consistency Modelscited by 1,778
  9. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  10. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  11. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
  12. 2021Variational Diffusion Modelscited by 1,145
31 more
  1. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 951
  2. 2023A General Theoretical Paradigm to Understand Learning from Human Preferencescited by 906
  3. 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Successcited by 844
  4. 2024DoRA: Weight-Decomposed Low-Rank Adaptationcited by 740
  5. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
  6. 2024RDT-1B: a Diffusion Foundation Model for Bimanual Manipulationcited by 670
  7. 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 564
  8. 2024ORPO: Monolithic Preference Optimization without Reference Modelcited by 556
  9. 2025SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Modelcited by 502
  10. 2025CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Modelscited by 440
  11. 2024Robotic Control via Embodied Chain-of-Thought Reasoningcited by 340
  12. 2024TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulationcited by 331
  13. 2024Simplifying, Stabilizing and Scaling Continuous-Time Consistency Modelscited by 276
  14. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 274
  15. 2023QA-LoRA: Quantization-Aware Low-Rank Adaptation of Large Language Modelscited by 154
  16. 2023PaLI-3 Vision Language Models: Smaller, Faster, Strongercited by 125
  17. 2025ChatVLA: Unified Multimodal Understanding and Robot Control with Vision-Language-Action Modelcited by 112
  18. 2024GRAPE: Generalizing Robot Policy via Preference Alignmentcited by 106
  19. 2024RoboMamba: Efficient Vision-Language-Action Model for Robotic Reasoning and Manipulationcited by 102
  20. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applicationscited by 82
  21. 2023Multi-View Masked World Models for Visual Robotic Manipulationcited by 78
  22. 2023Siamese Masked Autoencoderscited by 75
  23. 2024KVSharer: Efficient Inference via Layer-Wise Dissimilar KV Cache Sharingcited by 38
  24. 2025MoFlow: One-Step Flow Matching for Human Trajectory Forecasting via Implicit Maximum Likelihood Estimation based Distillationcited by 31
  25. 2024ReVLA: Reverting Visual Domain Limitation of Robotic Foundation Modelscited by 26
  26. 2024Ag2Manip: Learning Novel Manipulation Skills with Agent-Agnostic Visual and Action Representationscited by 23
  27. 2025ReBot: Scaling Robot Learning with Real-to-Sim-to-Real Robotic Video Synthesiscited by 22
  28. 2025FPC-VLA: A Vision-Language-Action Framework with a Supervisor for Failure Prediction and Correctioncited by 21
  29. 2026SnapFlow: One-Step Action Generation for Flow-Matching VLAs via Progressive Self-Distillationcited by 12
  30. 2026VLA-ATTC: Adaptive Test-Time Compute for VLA Models with Relative Action Critic Modelcited by 9
  31. 2026Sentinel-VLA: A Metacognitive VLA Model with Active Status Monitoring for Dynamic Reasoning and Error Recoverycited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026VLM4VLA: Revisiting Vision-Language-Models in Vision-Language-Action Models
  2. 2025VLA-R1: Enhancing Reasoning in Vision-Language-Action Models
  3. 2026Revisiting Parameter Redundancy in Vision-Language-Action Models: Insights from VLM-to-VLA Adaptation
  4. 2025VLA-Adapter: An Effective Paradigm for Tiny-Scale Vision-Language-Action Model
  5. 2026Fewer Steps, Better Actions: Rethinking Flow-Matching Inference for VLA Policies
  6. 2026Look Before You Leap: Distilling Tree Search into Action Evaluation for Frozen VLA Models
  7. 2026DyGRO-VLA: Cross-Task Scaling of Vision-Language-Action Models via Dynamic Grouped Residual Optimization
  8. 2025NORA-1.5: A Vision-Language-Action Model Trained using World Model- and Action-based Preference Rewards