mlsanityData to 14 Oct 2026

BitMar: Low-Bit Multimodal Fusion with Episodic Memory for Edge Devices

arXiv 2510.10560 ↗First posted 12 Oct 2025cs.CL cs.AI cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Visual token pruning and efficiency

Abstract (arXiv)

Cross-attention transformers and other multimodal vision-language models excel at grounding and generation; however, their extensive, full-precision backbones make it challenging to deploy them on edge devices. Memory-augmented architectures enhance the utilization of past context; however, most works rarely pair them with aggressive edge-oriented quantization. We introduce BitMar, a quantized multimodal transformer that proposes an external human-like episodic memory for effective image-text generation on hardware with limited resources. BitMar utilizes 1.58-bit encoders, one for text (BitNet-style) and one for vision (DiNOv2-based), to create compact embeddings that are combined and used to query a fixed-size key-value episodic memory. During vector retrieval, the BitNet decoder applies per-layer conditioning, which increases the contextual relevance of generated content. The decoder also employs attention sinks with a sliding-window mechanism to process long or streaming inputs under tight memory budgets. The combination of per-layer conditioning and sliding-window attention achieves a strong quality-speed trade-off, delivering competitive captioning and multimodal understanding at low latency with a small model footprint. These characteristics make BitMar well-suited for edge deployment.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 19 of its 26 references that are in the map. First posted on arXiv 12 Oct 2025.

Builds on

19 of 26 references.

References · 19

  1. 2017Attention Is All You Needcited by 60,361
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  4. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
  5. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  6. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
  7. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,462
  8. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,312
  9. 2021Improving language models by retrieving from trillions of tokenscited by 1,293
  10. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,089
  11. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,079
  12. 20218-bit Optimizers via Block-wise Quantizationcited by 403
7 more
  1. 2019Connecting Vision and Language with Localized Narrativescited by 226
  2. 2024MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Casescited by 223
  3. 2016Trained Ternary Quantizationcited by 208
  4. 2019And the Bit Goes Down: Revisiting the Quantization of Neural Networkscited by 78
  5. 2023TinyGPT-V: Efficient Multimodal Large Language Model via Small Backbonescited by 77
  6. 2025BabyLM Turns 3: Call for papers for the 2025 BabyLM workshopcited by 24
  7. 2024TinyLLM: A Framework for Training and Deploying Language Models at the Edge Computerscited by 9

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond Independent Optimization: Compression, MoE Routing, and Quantization Interactions in Multimodal Edge Intelligence
  2. 2026PixelUMM: Encoder-Free Unified Image and Video Understanding and Generation
  3. 2025HyperVL: An Efficient and Dynamic Multimodal Large Language Model for Edge Devices
  4. 2025Vision-Free Retrieval: Rethinking Multimodal Search with Textual Scene Descriptions
  5. 2026Cost-Efficient Multimodal LLM Inference via Cross-Tier GPU Heterogeneity
  6. 2026Argus-Unified: Towards A Compact and Economical Unified Model for Image Understanding and Generation
  7. 2025InfiniteVL: Synergizing Linear and Sparse Attention for Highly-Efficient, Unlimited-Input Vision-Language Models
  8. 2026Efficient Quantization-Aware Distillation with Cross-Modal Alignment for Edge Vision-Language Models