mlsanityData to 7 Oct 2026

RoboMamba: Efficient Vision-Language-Action Model for Robotic Reasoning and Manipulation

arXiv 2406.04339 ↗First posted 6 Jun 2024cs.CVcited by 102

On the map: Vision-language-action models → Generalist VLA models and benchmarks

also in 1 more
Abstract (arXiv)

A fundamental objective in robot manipulation is to enable models to comprehend visual scenes and execute actions. Although existing Vision-Language-Action (VLA) models for robots can handle a range of basic tasks, they still face challenges in two areas: (1) insufficient reasoning ability to tackle complex tasks, and (2) high computational costs for VLA model fine-tuning and inference. The recently proposed state space model (SSM) known as Mamba demonstrates promising capabilities in non-trivial sequence modeling with linear inference complexity. Inspired by this, we introduce RoboMamba, an end-to-end robotic VLA model that leverages Mamba to deliver both robotic reasoning and action capabilities, while maintaining efficient fine-tuning and inference. Specifically, we first integrate the vision encoder with Mamba, aligning visual tokens with language embedding through co-training, empowering our model with visual common sense and robotic-related reasoning. To further equip RoboMamba with SE(3) pose prediction abilities, we explore an efficient fine-tuning strategy with a simple policy head. We find that once RoboMamba possesses sufficient reasoning capability, it can acquire manipulation skills with minimal fine-tuning parameters (0.1% of the model) and time. In experiments, RoboMamba demonstrates outstanding reasoning capabilities on general and robotic evaluation benchmarks. Meanwhile, our model showcases impressive pose prediction results in both simulation and real-world experiments, achieving inference speeds 3 times faster than existing VLA models. Our project web page: https://sites.google.com/view/robomamba-web

Key facts. Cited by 102 papers in the map (data through 7 Oct 2026). Placed in Generalist VLA models and benchmarks, one of 13 directions in Vision-language-action models. Builds on 79 of its 87 references that are in the map. First posted on arXiv 6 Jun 2024.

Builds on

79 of 87 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control28 Jul

Other references · 78

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2023Visual Instruction Tuningcited by 7,792
  6. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  7. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  8. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  9. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  10. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  11. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  12. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
38 more
  1. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  2. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  3. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  4. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
  5. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  6. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
  7. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  8. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  9. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,696
  10. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
  11. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
  12. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,114
  13. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,000
  14. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 897
  15. 2018VizWiz Grand Challenge: Answering Visual Questions from Blind Peoplecited by 852
  16. 2023RWKV: Reinventing RNNs for the Transformer Eracited by 814
  17. 2022Simplified State Space Layers for Sequence Modelingcited by 776
  18. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 776
  19. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 733
  20. 2023ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoderscited by 703
  21. 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledgecited by 702
  22. 2023MiniGPT-v2: large language model as a unified interface for vision-language multi-task learningcited by 617
  23. 2021Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layerscited by 607
  24. 2024TinyLlama: An Open-Source Small Language Modelcited by 604
  25. 2023LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Modelcited by 602
  26. 2023Retentive Network: A Successor to Transformer for Large Language Modelscited by 594
  27. 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 584
  28. 2024MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?cited by 573
  29. 2023OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Modelscited by 560
  30. 2018PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understandingcited by 521
  31. 2022Hungry Hungry Hippos: Towards Language Modeling with State Space Modelscited by 495
  32. 2020TextCaps: a Dataset for Image Captioning with Reading Comprehensioncited by 398
  33. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 359
  34. 2024MambaIR: A Simple Baseline for Image Restoration with State-Space Modelcited by 349
  35. 2023GPT-Driver: Learning to Drive with GPTcited by 344
  36. 2023Vision-Language Foundation Models as Effective Robot Imitatorscited by 344
  37. 2021XCiT: Cross-Covariance Image Transformerscited by 335
  38. 2024VM-UNet: Vision Mamba UNet for Medical Image Segmentationcited by 286

50 most cited of 78 All 78 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 48; 2026: 49.

102 papers.

Sort:

By area:

  1. 2024RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulationcited by 166
  2. 2025EfficientVLA: Training-Free Acceleration and Compression for Vision-Language-Action Modelscited by 89
  3. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applicationscited by 82
  4. 2025Tarsier2: Advancing Large Vision-Language Models from Detailed Video Description to Comprehensive Video Understandingcited by 80
  5. 2025A Survey on Vision-Language-Action Models for Autonomous Drivingcited by 72
  6. 2025MoLe-VLA: Dynamic Layer-skipping Vision Language Action Model via Mixture-of-Layers for Efficient Robot Manipulationcited by 69
  7. 2025Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Surveycited by 66
  8. 2025WorldEval: World Model as Real-World Robot Policies Evaluatorcited by 64
  9. 2025ChatVLA-2: Vision-Language-Action Model with Open-World Embodied Reasoning from Pretrained Knowledgecited by 49
  10. 2025Pure Vision Language Action (VLA) Models: A Comprehensive Surveycited by 46
  11. 2025Humanoid-VLA: Towards Universal Humanoid Control with Visual Integrationcited by 43
  12. 2024A Survey of Mambacited by 39
38 more
  1. 2025BadVLA: Towards Backdoor Attacks on Vision-Language-Action Models via Objective-Decoupled Optimizationcited by 37
  2. 2025Planning with Reasoning using Vision Language World Modelcited by 36
  3. 2025A Survey on Efficient Vision-Language-Action Modelscited by 34
  4. 2025Exploring the Limits of Vision-Language-Action Manipulations in Cross-task Generalizationcited by 31
  5. 2025Spec-VLA: Speculative Decoding for Vision-Language-Action Models with Relaxed Acceptancecited by 31
  6. 2025Think Twice, Act Once: Token-Aware Compression and Action Reuse for Efficient Inference in Vision-Language-Action Modelscited by 30
  7. 2025Agentic Robot: A Brain-Inspired Framework for Vision-Language-Action Models in Embodied Agentscited by 30
  8. 2025TIGeR: Tool-Integrated Geometric Reasoning in Vision-Language Models for Roboticscited by 27
  9. 2025ManipLVM-R1: Reinforcement Learning for Reasoning in Embodied Manipulation with Large Vision-Language Modelscited by 26
  10. 2025SemanticVLA: Semantic-Aligned Sparsification and Enhancement for Efficient Robotic Manipulationcited by 20
  11. 2026FASTER: Rethinking Real-Time Flow VLAscited by 20
  12. 2025Parallels Between VLA Model Post-Training and Human Motor Learning: Progress, Challenges, and Trendscited by 17
  13. 2025Chain-of-Action: Trajectory Autoregressive Modeling for Robotic Manipulationcited by 16
  14. 2025RoboDexVLM: Visual Language Model-Enabled Task Planning and Motion Control for Dexterous Robot Manipulationcited by 15
  15. 2026On-the-Fly VLA Adaptation via Test-Time Reinforcement Learningcited by 14
  16. 2025MinD: Learning A Dual-System World Model for Real-Time Planning and Implicit Risk Analysiscited by 13
  17. 2025RationalVLA: A Rational Vision-Language-Action Model with Dual Systemcited by 12
  18. 2025GP3: A 3D Geometry-Aware Policy with Multi-View Images for Robotic Manipulationcited by 11
  19. 2025RoboChemist: Long-Horizon and Safety-Compliant Robotic Chemical Experimentationcited by 9
  20. 2026TurboVLA: Real-Time Vision-Language-Action Model at 32 Hz on an RTX 4090 with <1 GB VRAMcited by 9
  21. 2024Mamba as a motion encoder for robotic imitation learningcited by 7
  22. 2025Survey of Vision-Language-Action Models for Embodied Manipulationcited by 7
  23. 2026ReMoRa: Multimodal Large Language Model based on Refined Motion Representation for Long-Video Understandingcited by 7
  24. 2026Gaze-Regularized Vision-Language-Action Models for Robotic Manipulationcited by 7
  25. 2025Focusing on What Matters: Object-Agent-centric Tokenization for Vision Language Action modelscited by 6
  26. 2025ELMUR: External Layer Memory with Update/Rewrite for Long-Horizon RL Problemscited by 6
  27. 2025ActionFlow: A Pipelined Action Acceleration for Vision Language Models on Edgecited by 6
  28. 2026SaPaVe: Towards Active Perception and Manipulation in Vision-Language-Action Models for Roboticscited by 6
  29. 2026Characterizing Vision-Language-Action Models across XPUs: Constraints and Acceleration for On-Robot Deploymentcited by 6
  30. 2025Learning Primitive Embodied World Models: Towards Scalable Robotic Learningcited by 5
  31. 2026TIDAL: Temporally Interleaved Diffusion and Action Loop for High-Frequency VLA Controlcited by 5
  32. 2026TwinRL: Digital Twin-Driven Reinforcement Learning for Real-World Robotic Manipulationcited by 5
  33. 2026From Abstraction to Instantiation: Learning Behavioral Representation for Vision-Language-Action Modelcited by 5
  34. 2025Towards Reliable LLM-based Robot Planning via Combined Uncertainty Estimationcited by 4
  35. 2025ActDistill: General Action-Guided Self-Derived Distillation for Efficient Vision-Language-Action Modelscited by 4
  36. 2025An Anatomy of Vision-Language-Action Models: From Modules to Milestones and Challengescited by 4
  37. 2025OMP: One-step Meanflow Policy with Directional Alignmentcited by 4
  38. 2026Spatial Memory for Out-of-Vision Manipulation in Vision-Language-Actioncited by 4

50 most cited of 102 All 102 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026RoboSPA: Can VLA Models Go Beyond Simple Scenes and Short-Horizon Tasks?
  2. 2024RoboUniView: Visual-Language Model with Unified View Representation for Robotic Manipulation
  3. 2025From Seeing to Doing: Bridging Reasoning and Decision for Robotic Manipulation
  4. 2025Efficient Vision-Language-Action Models for Embodied Manipulation: A Systematic Survey
  5. 2026RoboSemanticBench: Diagnosing Semantic Grounding in Action Prediction for VLA Models
  6. 2025Seeing Across Views: Benchmarking Spatial Reasoning of Vision-Language Models in Robotic Scenes
  7. 2024TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulation
  8. 2026RoboInter: A Holistic Intermediate Representation Suite Towards Robotic Manipulation