M2-VLA: Boosting Vision-Language Models for Generalizable Manipulation via Layer Mixture and Meta-Skills
On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models
also in 1 more
Abstract (arXiv)
Current Vision-Language-Action (VLA) models predominantly rely on end-to-end fine-tuning. While effective, this paradigm compromises the inherent generalization capabilities of Vision-Language Models (VLMs) and incurs catastrophic forgetting. To address these limitations, we propose M2-VLA, which demonstrates that a generalized VLM is able to serve as a powerful backbone for robotic manipulation directly. However, it remains a key challenge to bridge the gap between the high-level semantic understanding of VLMs and the precise requirements of robotic control. To overcome this, we introduce the Mixture of Layers (MoL) strategy that selectively extracts task-critical information from dense semantic features. Furthermore, to facilitate efficient trajectory learning under constrained model capacity, we propose a Meta Skill Module (MSM) that integrates strong inductive biases. Extensive experiments in both simulated and real-world environments demonstrate the effectiveness of our approach. Furthermore, generalization and ablation studies validate the architecture's zero-shot capabilities and confirm the contribution of each key component. Our code and pre-trained models will be made publicly available.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 37 of its 41 references that are in the map. First posted on arXiv 27 Apr 2026.
Builds on
37 of 41 references.
Closest ancestors · 6
Same direction, newest first
- 2025DepthLM: Metric Depth From Vision Language Models29 Sep
- 2025VLA-Adapter: An Effective Paradigm for Tiny-Scale Vision-Language-Action Model11 Sep
- 2025SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics2 Jun
- 2025VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstruction26 May
- 2025CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models27 Mar
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control28 Jul
Other references · 31
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2023Visual Instruction Tuningcited by 7,792
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
- 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
- 2022Training Compute-Optimal Large Language Modelscited by 2,927
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
- 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
- 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
- 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
- 2022A Generalist Agentcited by 802
- 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 776
- 2024RDT-1B: a Diffusion Foundation Model for Bimanual Manipulationcited by 670
19 more
- 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 622
- 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 564
- 2024Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Modelscited by 336
- 2025MolmoAct: Action Reasoning Models that can Reason in Spacecited by 189
- 2024Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learningcited by 153
- 2025ThinkAct: Vision-Language-Action Reasoning via Reinforced Visual Latent Planningcited by 152
- 2025ReconVLA: Reconstructive Vision-Language-Action Model as Effective Robot Perceivercited by 80
- 2025OpenHelix: A Short Survey, Empirical Analysis, and Open-Source Dual-System VLA Model for Robotic Manipulationcited by 79
- 2025Eagle 2: Building Post-Training Data Strategies from Scratch for Frontier Vision-Language Modelscited by 69
- 2025FlowVLA: Visual Chain of Thought-based Motion Reasoning for Vision-Language-Action Modelscited by 50
- 2025FlyLoRA: Boosting Task Decoupling and Parameter Efficiency via Implicit Rank-Wise Mixture-of-Expertscited by 34
- 2025A Survey on Efficient Vision-Language-Action Modelscited by 34
- 2025VGGT-DP: Generalizable Robot Control via Vision Foundation Modelscited by 11
- 2025Rethinking Latent Redundancy in Behavior Cloning: An Information Bottleneck Approach for Robot Manipulationcited by 10
- 2024Bayesian Parameter-Efficient Fine-Tuning for Overcoming Catastrophic Forgettingcited by 9
- 2025GraLoRA: Granular Low-Rank Adaptation for Parameter-Efficient Fine-Tuningcited by 8
- 2024Waypoint-Based Reinforcement Learning for Robot Manipulation Taskscited by 7
- 2025RoboWheel: A Data Engine from Real-World Human Demonstrations for Cross-Embodiment Robotic Learningcited by 5
- 202510 Open Challenges Steering the Future of Vision-Language-Action Modelscited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulation
- 2026V-Link: Recovering Lost Visual Representations in Action DiT for Vision-Language-Action Models
- 2025Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Survey
- 2025MoLe-VLA: Dynamic Layer-skipping Vision Language Action Model via Mixture-of-Layers for Efficient Robot Manipulation
- 2026AC^2-VLA: Action-Context-Aware Adaptive Computation in Vision-Language-Action Models for Efficient Robotic Manipulation
- 2026GT-VLA: Target-Conditioned Trace Guidance for Generalizable Robotic Manipulation
- 2026Look Before Acting: Enhancing Vision Foundation Representations for Vision-Language-Action Models
- 2025MoS-VLA: A Vision-Language-Action Model with One-Shot Skill Adaptation