# Video understanding with MLLMs Area: [Multimodal LLMs / vision-language models](https://mlsanity.com/mllm/index.md). HTML: https://mlsanity.com/mllm/video-understanding-with-mllms/ ## Key facts - Video understanding with MLLMs: 742 papers on arXiv since 2022 (data through 14 Oct 2026). - 147 of its 742 mapped papers (20%) cite “Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis” (2024, arXiv 2405.21075). - 112 of its 742 mapped papers (15%) cite “MVBench: A Comprehensive Multi-modal Video Understanding Benchmark” (2023, arXiv 2311.17005). - 34 new papers 17 Sep–14 Oct. ## Most cited here - [Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis](https://mlsanity.com/paper/2405.21075/) (2024), cited by 147 papers here - [MVBench: A Comprehensive Multi-modal Video Understanding Benchmark](https://mlsanity.com/paper/2311.17005/) (2023), cited by 112 papers here - [Video-LLaVA: Learning United Visual Representation by Alignment Before Projection](https://mlsanity.com/paper/2311.10122/) (2023), cited by 97 papers here - [Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding](https://mlsanity.com/paper/2306.02858/) (2023), cited by 92 papers here - [LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding](https://mlsanity.com/paper/2407.15754/) (2024), cited by 81 papers here - [LVBench: An Extreme Long Video Understanding Benchmark](https://mlsanity.com/paper/2406.08035/) (2024), cited by 57 papers here - [Long Context Transfer from Language to Vision](https://mlsanity.com/paper/2406.16852/) (2024), cited by 54 papers here - [LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models](https://mlsanity.com/paper/2311.17043/) (2023), cited by 53 papers here - [Video-R1: Reinforcing Video Reasoning in MLLMs](https://mlsanity.com/paper/2503.21776/) (2025), cited by 53 papers here - [TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding](https://mlsanity.com/paper/2312.02051/) (2023), cited by 50 papers here ## New (17 Sep–14 Oct) - [OmniCompass: A Holistic Benchmark for Omni-Modal Video Grounding and Understanding](https://mlsanity.com/paper/2610.18829/) (2026-10-13) - [When Left Is Right? Camera-Motion Understanding beyond Screen](https://mlsanity.com/paper/2610.17242/) (2026-10-12) - [EventLens: Event-Structure Reinforcement Learning for Video Understanding](https://mlsanity.com/paper/2610.16172/) (2026-10-12) - [Elephant in the Fridge: Constant-Memory Frame Packing for Long Video Understanding](https://mlsanity.com/paper/2610.16118/) (2026-10-12) - [PhySimCode: A Benchmark and Evaluation Method for Physics Video to Code Generation](https://mlsanity.com/paper/2610.15722/) (2026-10-11) - [TIDE: Trajectory-Informed Skill Distillation for Efficient Text-to-Video Prompt Optimization](https://mlsanity.com/paper/2610.15098/) (2026-10-10) - [OmniCapBench: A Deep-Structured Evaluation Framework for Fine-Grained Audio-Visual Captioning](https://mlsanity.com/paper/2610.12458/) (2026-10-08) - [VepAgent: Bridging Causal-Transition via Tool-Augmented Reinforcement Learning for Video Event Prediction](https://mlsanity.com/paper/2610.06293/) (2026-10-05) - [ReMem: Streaming Video Understanding With Long Context Retention](https://mlsanity.com/paper/2610.05940/) (2026-10-05) - [Representation--Behavior Alignment for Explainable Weakly-Supervised Video Anomaly Detection](https://mlsanity.com/paper/2610.05129/) (2026-10-04) - [Rethinking Long-Video Efficiency: A Joint Allocation Perspective on Frames, Pixels, and Front-End Latency](https://mlsanity.com/paper/2610.04318/) (2026-10-03) - [Behavior Pack Optimization for Video MLLM Post-Training](https://mlsanity.com/paper/2610.03141/) (2026-10-02) - [FlashBack: Knowing When to Remember in Streaming Vision-Language Models](https://mlsanity.com/paper/2610.01192/) (2026-10-01) - [Paying for Too Many Tokens? Valid and Cost-Efficient Multimodal LLM Annotation with Simple Heuristics](https://mlsanity.com/paper/2610.00809/) (2026-09-30) - [FORTE: Adaptive Scoring and Exact Keyframe Selection for Long-Video Question Answering](https://mlsanity.com/paper/2610.00573/) (2026-09-30) - [Less Data, Better Timing: Student-Curriculum Coupling for VLM On-Policy Distillation in Temporal Video Grounding](https://mlsanity.com/paper/2609.40055/) (2026-09-30) - [Frame Differential On-Policy Self-Distillation for Video Reasoning](https://mlsanity.com/paper/2609.39021/) (2026-09-30) - [PixelUMM: Encoder-Free Unified Image and Video Understanding and Generation](https://mlsanity.com/paper/2609.38597/) (2026-09-29) - [ThinkV2V: Unleashing the Reasoning Capability of MLLMs for Instruction-Guided Video Editing](https://mlsanity.com/paper/2609.38541/) (2026-09-29) - [VidHarness: Evolving Agent Harnesses for Cost-Efficient Long Video Understanding](https://mlsanity.com/paper/2609.38413/) (2026-09-29) - [LazySloth: Bounded LLM-based Lazy Tree Search for Fast Long Video Comprehension](https://mlsanity.com/paper/2609.37426/) (2026-09-29) - [Beyond Binary Preferences: Graded Preference Optimization for Limb-Motion Captioning](https://mlsanity.com/paper/2609.36628/) (2026-09-29) - [MemEvo: Automatic Discovery of Streaming Video Memory Mechanisms](https://mlsanity.com/paper/2609.36581/) (2026-09-29) - [Counterfactual Attention Policy Distillation for Temporal Video Grounding](https://mlsanity.com/paper/2609.34581/) (2026-09-29) - [PARSEE-VAD: Efficient Training-Free Online Video Anomaly Detection via Proposition-Aware Reasoning and Streaming Evidence Escalation](https://mlsanity.com/paper/2609.33236/) (2026-09-29) - [ReVA: A Scene-Centric Dataset Beyond Repetition for Remote Sensing Video Question Answering](https://mlsanity.com/paper/2609.35507/) (2026-09-28) - [Sprout: Building Dynamic Memory While Reasoning for Agentic Video Understanding](https://mlsanity.com/paper/2609.35497/) (2026-09-28) - [ActionLens: Diagnosing Spatial-Temporal Binding Failures in Vision-Language Models](https://mlsanity.com/paper/2609.34547/) (2026-09-28) - [MetaSampling: Making Frame Samplers Efficient for Long-Video Question Answering](https://mlsanity.com/paper/2609.33998/) (2026-09-27) - [KeyRec: Bounded Visual Memory for Streaming and Long-Video Understanding](https://mlsanity.com/paper/2609.32182/) (2026-09-26)