mlsanityData to 14 Oct 2026

LCMF: Lightweight Cross-Modality Mambaformer for Embodied Robotics VQA

arXiv 2509.18576 ↗First posted 23 Sep 2025cs.RO cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Multimodal semantic learning plays a critical role in embodied intelligence, especially when robots perceive their surroundings, understand human instructions, and make intelligent decisions. However, the field faces technical challenges such as effective fusion of heterogeneous data and computational efficiency in resource-constrained environments. To address these challenges, this study proposes the lightweight LCMF cascaded attention framework, introducing a multi-level cross-modal parameter sharing mechanism into the Mamba module. By integrating the advantages of Cross-Attention and Selective parameter-sharing State Space Models (SSMs), the framework achieves efficient fusion of heterogeneous modalities and semantic complementary alignment. Experimental results show that LCMF surpasses existing multimodal baselines with an accuracy of 74.29% in VQA tasks and achieves competitive mid-tier performance within the distribution cluster of Large Language Model Agents (LLM Agents) in EQA video tasks. Its lightweight design achieves a 4.35-fold reduction in FLOPs relative to the average of comparable baselines while using only 166.51M parameters (image-text) and 219M parameters (video-text), providing an efficient solution for Human-Robot Interaction (HRI) applications in resource-constrained scenarios with strong multimodal decision generalization capabilities.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 27 of its 35 references that are in the map. First posted on arXiv 23 Sep 2025.

Builds on

27 of 35 references.

References · 27

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  4. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  5. 2015VQA: Visual Question Answeringcited by 2,506
  6. 2017Non-local Neural Networkscited by 2,293
  7. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
  8. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,553
  9. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,358
  10. 2021CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classificationcited by 806
  11. 2019Unified Vision-Language Pre-Training for Image Captioning and VQAcited by 494
  12. 2022LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Modelscited by 462
15 more
  1. 2019Deep Modular Co-Attention Networks for Visual Question Answeringcited by 339
  2. 2017Embodied Question Answeringcited by 336
  3. 2022Context Autoencoder for Self-Supervised Representation Learningcited by 297
  4. 2023EmbodiedGPT: Vision-Language Pre-Training via Embodied Chain of Thoughtcited by 279
  5. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 279
  6. 2023Foundation Models in Robotics: Applications, Challenges, and the Futurecited by 216
  7. 2023A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trendscited by 207
  8. 2024Scene-LLM: Extending Language Model for 3D Visual Understanding and Reasoningcited by 175
  9. 2021Embodied Intelligence via Learning and Evolutioncited by 163
  10. 2024VL-Mamba: Exploring State Space Models for Multimodal Learningcited by 93
  11. 2024NavCoT: Boosting LLM-Based Vision-and-Language Navigation via Learning Disentangled Reasoningcited by 79
  12. 2020Parameter Efficient Multimodal Transformers for Video Representation Learningcited by 52
  13. 2023Masked Modeling for Self-supervised Representation Learning on Vision and Beyondcited by 13
  14. 2019Embodiment in Socially Interactive Robotscited by 11
  15. 2024Interactive Masked Image Modeling for Multimodal Object Detection in Remote Sensingcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025M3ET: Efficient Vision-Language Learning for Robotics based on Multimodal Mamba-Enhanced Transformer
  2. 2024AlignMamba: Enhancing Multimodal Mamba with Local and Global Cross-modal Alignment
  3. 2026Query-based Cross-Modal Projector Bolstering Mamba Multimodal LLM
  4. 2024Large Language Models for Robotics: Opportunities, Challenges, and Perspectives
  5. 2024From Linguistic Giants to Sensory Maestros: A Survey on Cross-Modal Reasoning with Large Language Models
  6. 2025End-to-End Multi-Modal Diffusion Mamba
  7. 2024Cobra: Extending Mamba to Multi-Modal Large Language Model for Efficient Inference
  8. 2024CROME: Cross-Modal Adapters for Efficient Multimodal LLM