Towards Universal Video MLLMs with Attribute-Structured and Quality-Verified Instructions
Not on the map. Why?
Abstract (arXiv)
Universal video understanding requires modeling fine-grained visual and audio information over time in diverse real-world scenarios. However, the performance of existing models is primarily constrained by video-instruction data that represents complex audiovisual content as single, incomplete descriptions, lacking fine-grained organization and reliable annotation. To address this, we introduce: (i) ASID-1M, an open-source collection of one million structured, fine-grained audiovisual instruction annotations with single- and multi-attribute supervision; (ii) ASID-Verify, a scalable data curation pipeline for annotation, with automatic verification and refinement that enforces semantic and temporal consistency between descriptions and the corresponding audiovisual content; and (iii) ASID-Captioner, a video understanding model trained via Supervised Fine-Tuning (SFT) on the ASID-1M. Experiments across seven benchmarks covering audiovisual captioning, attribute-wise captioning, caption-based QA, and caption-based temporal grounding show that ASID-Captioner improves fine-grained caption quality while reducing hallucinations and improving instruction following. It achieves state-of-the-art performance among open-source models and is competitive with Gemini-3-Pro.
Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 46 of its 54 references that are in the map. First posted on arXiv 13 Feb 2026.
Builds on
46 of 54 references.
References · 46
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
- 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
- 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,461
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
- 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
- 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
- 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,047
- 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
34 more
- 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
- 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 946
- 2017Dense-Captioning Events in Videoscited by 789
- 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 749
- 2025Qwen2.5-Omni Technical Reportcited by 704
- 2017Towards Automatic Learning of Procedures from Web Instructional Videoscited by 616
- 2017TALL: Temporal Activity Localization via Language Querycited by 581
- 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 536
- 2024Long Context Transfer from Language to Visioncited by 502
- 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 488
- 2025Qwen3-Omni Technical Reportcited by 451
- 2024ShareGPT4Video: Improving Video Understanding and Generation with Better Captionscited by 386
- 2023WhisperX: Time-Accurate Speech Transcription of Long-Form Audiocited by 320
- 2024mPLUG-Owl3: Towards Long Image-Sequence Understanding in Multi-Modal Large Language Modelscited by 285
- 2025VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuningcited by 248
- 2024Pixtral 12Bcited by 174
- 2025WorldSense: Evaluating Real-world Omnimodal Understanding for Multimodal LLMscited by 147
- 2024Tarsier: Recipes for Training and Evaluating Large Video Description Modelscited by 140
- 2024AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmarkcited by 137
- 2024Aria: An Open Multimodal Native Mixture-of-Experts Modelcited by 136
- 2024video-SALMONN: Speech-Enhanced Audio-Visual Large Language Modelscited by 101
- 2025Daily-Omni: Towards Audio-Visual Reasoning with Temporal Alignment across Modalitiescited by 101
- 2025Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunitiescited by 81
- 2025Tarsier2: Advancing Large Vision-Language Models from Detailed Video Description to Comprehensive Video Understandingcited by 80
- 2025HumanOmniV2: From Understanding to Omni-Modal Reasoning with Contextcited by 50
- 2025ARC-Hunyuan-Video-7B: Structured Video Comprehension of Real-World Shortscited by 36
- 2025VideoCap-R1: Enhancing MLLMs for Video Captioning via Structured Thinkingcited by 27
- 2025Bee: A High-Quality Corpus and Full-Stack Suite to Unlock Advanced Fully Open MLLMscited by 27
- 2025AVoCaDO: An Audiovisual Video Captioner Driven by Temporal Orchestrationcited by 26
- 2025Omni-Captioner: Data Pipeline, Models, and Benchmark for Omni Detailed Perceptioncited by 25
- 2025VidCapBench: A Comprehensive Benchmark of Video Captioning for Controllable Text-to-Video Generationcited by 22
- 2025UGC-VideoCaptioner: An Omni UGC Video Detail Caption Model and New Benchmarkscited by 17
- 2025TempSamp-R1: Effective Temporal Sampling with Reinforcement Fine-Tuning for Video LLMscited by 15
- 2025IF-VidCap: Can Video Caption Models Follow Instructions?cited by 7
Built on by
Citing papers per year: 2026: 7.
7 papers.
- 2026Script-a-Video: Deep Structured Audio-visual Captions via Factorized Streams and Relational Groundingcited by 6
- 2026LatentOmni: Rethinking Omni-Modal Understanding via Unified Audio-Visual Latent Reasoningcited by 5
- 2026OmniVideo-100K: A Dataset for Audio-Visual Reasoning through Structured Scripts and Evidence Chainscited by 5
- 2026CapRiCorn-1K: A Comprehensive Benchmark for Video Captioning and Subject Referential Consistency Across Temporal Scalescited by 1
- 2026AVSCap: Orchestrating Audio-Visual Synergy for Omni-modal Video Captioningcited by 1
- 2026Reasoning for Social Audio-Visual Question Answering: Where Do We Stand?cited by 0
- 2026OmniCapBench: A Deep-Structured Evaluation Framework for Fine-Grained Audio-Visual Captioningcited by 0
- 2026OmniCapBench: A Deep-Structured Evaluation Framework for Fine-Grained Audio-Visual Captioning8 Oct 2026
- 2026Reasoning for Social Audio-Visual Question Answering: Where Do We Stand?13 Aug 2026
- 2026AVSCap: Orchestrating Audio-Visual Synergy for Omni-modal Video Captioning14 Jul 2026
- 2026CapRiCorn-1K: A Comprehensive Benchmark for Video Captioning and Subject Referential Consistency Across Temporal Scales20 Jun 2026
- 2026OmniVideo-100K: A Dataset for Audio-Visual Reasoning through Structured Scripts and Evidence Chains12 Jun 2026
- 2026LatentOmni: Rethinking Omni-Modal Understanding via Unified Audio-Visual Latent Reasoning21 May 2026
- 2026Script-a-Video: Deep Structured Audio-visual Captions via Factorized Streams and Relational Grounding13 Apr 2026
Similar, not cited
Close in wording, no citation either way
- 2026OmniCap-IF: Benchmarking and Improving Instruction Following Abilities for Omni-Video Captioning
- 2025OmniVideoBench: Towards Audio-Visual Understanding Evaluation for Omni MLLMs
- 2026Video-IFBench: Evaluating Instruction Following of Multimodal LLMs in Video Understanding Scenarios
- 2026DenseStep2M: A Scalable, Training-Free Pipeline for Dense Instructional Video Annotation
- 2025SF2T: Self-supervised Fragment Finetuning of Video-LLMs for Fine-Grained Understanding
- 2024Fewer Tokens and Fewer Videos: Extending Video Understanding Abilities in Large Vision-Language Models
- 2024Distilling Vision-Language Models on Millions of Videos
- 2025DisCo: Towards Distinct and Coherent Visual Encapsulation in Video MLLMs