mlsanityData to 14 Oct 2026

MOSS-VL Technical Report

arXiv 2608.15045 ↗First posted 15 Aug 2026cs.CVcited by 5

On the map: Multimodal LLMs / vision-language models → Video understanding with MLLMs

Abstract (arXiv)

We present MOSS-VL, an open vision-language model family that treats real-time interaction -- perceiving while it speaks -- as a first-class capability. It is co-designed across the stack: the language decoder attends to vision only through gated cross-attention, so the model can naturally see incoming frames while generating; a synthesized interaction corpus supervises when to speak, when to stay silent, and when to revise; and a staged curriculum concentrates all real-time-specific training in one light final stage over a strong offline foundation. Offline, MOSS-VL-Instruct is competitive at comparable scale and leads temporal-reasoning video sets. Across four streaming benchmarks, MOSS-VL-Realtime posts the best average on three (second on the fourth) among open-source streaming models, sweeping the three subsets that squarely test proactive behavior -- 66.0 vs. 37.5 for the best baseline on OmniMMI Proactive Alerting. With 11.3B parameters but visual tokens outside the decoded sequence, MOSS-VL widens its time-to-first-token advantage over same-backbone Qwen3-VL-8B from 2.8x to 5.1x as visual context grows. We release all five checkpoints, the training curriculum, and the real-time inference code at https://github.com/OpenMOSS/MOSS-VL.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 53 of its 60 references that are in the map. First posted on arXiv 15 Aug 2026.

Builds on

53 of 60 references.

Closest ancestors · 12

Same direction, newest first

  1. 2026Video-MME-Logical: A Controlled Diagnostic Benchmark for Video Temporal-Logical Reasoning26 Jun
  2. 2026ROMA: Real-time Omni-Multimodal Assistant with Interactive Streaming Understanding15 Jan
  3. 2025TimeLens: Rethinking Video Temporal Grounding with Multimodal LLMs16 Dec
  4. 2025MMDuet2: Enhancing Proactive Interaction of Video MLLMs with Multi-Turn Reinforcement Learning7 Dec
  5. 2025MINERVA: Evaluating Complex Video Reasoning1 May
  6. 2025OmniMMI: A Comprehensive Multi-modal Interaction Benchmark in Streaming Video Contexts29 Mar
  7. 2025ViSpeak: Visual Instruction Feedback in Streaming Videos17 Mar
  8. 2024StreamingBench: Assessing the Gap for MLLMs to Achieve Streaming Video Understanding6 Nov
  9. 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding22 Jul
  10. 2024LVBench: An Extreme Long Video Understanding Benchmark12 Jun
  11. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis31 May
  12. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmark28 Nov

Other references · 41

  1. 2025Qwen3 Technical Reportcited by 8,820
  2. 2025Qwen2.5-VL Technical Reportcited by 5,587
  3. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  4. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  5. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  6. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
  7. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,729
  8. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
  9. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  10. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  11. 2016Modeling Context in Referring Expressionscited by 1,063
  12. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 914
29 more
  1. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 838
  2. 2016A Diagram Is Worth A Dozen Imagescited by 804
  3. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 680
  4. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 611
  5. 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 550
  6. 2024FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precisioncited by 519
  7. 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 482
  8. 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 476
  9. 2024MLVU: Benchmarking Multi-task Long Video Understandingcited by 292
  10. 2024MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans?cited by 292
  11. 2026Gemma 4 Technical Reportcited by 259
  12. 2023Teaching CLIP to Count to Tencited by 198
  13. 2024MuirBench: A Comprehensive Benchmark for Robust Multi-image Understandingcited by 188
  14. 2024OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoningcited by 173
  15. 2024OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotationscited by 150
  16. 2025OVO-Bench: How Far is Your Video-LLMs from Real-World Online Video Understanding?cited by 142
  17. 2024EmbSpatial-Bench: Benchmarking Spatial Understanding for Embodied Tasks with Large Vision-Language Modelscited by 107
  18. 2025VisuLogic: A Benchmark for Evaluating Visual Reasoning in Multi-modal Large Language Modelscited by 107
  19. 2024TOMATO: Assessing Visual Temporal Reasoning Capabilities in Multimodal Foundation Modelscited by 73
  20. 2026Video-MME-v2: Towards the Next Stage in Benchmarks for Comprehensive Video Understandingcited by 55
  21. 2024VideoLLM Knows When to Speak: Enhancing Time-Sensitive Video Comprehension with Video-Text Duet Interaction Formatcited by 47
  22. 2026LLaVA-OneVision-2: Towards Next-Generation Perceptual Intelligencecited by 39
  23. 2020Words aren't enough, their order matters: On the Robustness of Grounding Visual Referring Expressionscited by 33
  24. 2025ProactiveVideoQA: A Comprehensive Benchmark Evaluating Proactive Interactions in Video Large Language Modelscited by 27
  25. 2025Streaming Video Instruction Tuningcited by 26
  26. 2026AURA: Always-On Understanding and Real-Time Assistance via Video Streamscited by 16
  27. 2026JoyAI-VL-Interaction: Real-Time Vision-Language Interaction Intelligencecited by 14
  28. 2026VideoChat3: Fully Open Video MLLM for Efficient and Generalist Video Understandingcited by 10
  29. 2026MOSS-Video-Preview: Toward Real-Time Video Understanding via Cross-Attentioncited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026VSAS-Bench: Real-Time Evaluation of Visual Streaming Assistant Models
  2. 2026Harnessing Streaming Video in the Wild
  3. 2024VLsI: Verbalized Layers-to-Interactions from Large to Small Vision Language Models
  4. 2025StreamingVLM: Real-Time Understanding for Infinite Video Streams
  5. 2026Instruction-Evidence Contrastive Dual-Stream Decoding for Grounded Vision-Language Reasoning
  6. 2024Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Models
  7. 2026Act2See: Emergent Active Visual Perception for Video Reasoning
  8. 2023MIMIC-IT: Multi-Modal In-Context Instruction Tuning