# SlowFast-LLaVA-1.5: A Family of Token-Efficient Video Large Language Models for Long-Form Video Understanding

arXiv: https://arxiv.org/abs/2503.18943 · HTML: https://mlsanity.com/paper/2503.18943/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 128 of its 148 references that are in the map.
- First posted on arXiv 24 Mar 2025.

## Builds on (most cited first, 30 of 128)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [Sigmoid Loss for Language Image Pre-Training](https://mlsanity.com/paper/2303.15343/) (2023)
- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI](https://mlsanity.com/paper/2311.16502/) (2023)
- [Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering](https://mlsanity.com/paper/2209.09513/) (2022)
- [Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone](https://mlsanity.com/paper/2404.14219/) (2024)
- [Towards VQA Models That Can Read](https://mlsanity.com/paper/1904.08920/) (2019)
- [MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts](https://mlsanity.com/paper/2310.02255/) (2023)
- [Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling](https://mlsanity.com/paper/2412.05271/) (2024)
- [ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning](https://mlsanity.com/paper/2203.10244/) (2022)
- [Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding](https://mlsanity.com/paper/2306.02858/) (2023)
- [CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning](https://mlsanity.com/paper/1612.06890/) (2016)
- [Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis](https://mlsanity.com/paper/2405.21075/) (2024)
- [Video-LLaVA: Learning United Visual Representation by Alignment Before Projection](https://mlsanity.com/paper/2311.10122/) (2023)
- [DocVQA: A Dataset for VQA on Document Images](https://mlsanity.com/paper/2007.00398/) (2020)
- [MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities](https://mlsanity.com/paper/2308.02490/) (2023)
- [MVBench: A Comprehensive Multi-modal Video Understanding Benchmark](https://mlsanity.com/paper/2311.17005/) (2023)
- [Perceiver: General Perception with Iterative Attention](https://mlsanity.com/paper/2103.03206/) (2021)
- [MiniCPM-V: A GPT-4V Level MLLM on Your Phone](https://mlsanity.com/paper/2408.01800/) (2024)
- [Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Models](https://mlsanity.com/paper/2306.05424/) (2023)
- [How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites](https://mlsanity.com/paper/2404.16821/) (2024)
- [ShareGPT4V: Improving Large Multi-Modal Models with Better Captions](https://mlsanity.com/paper/2311.12793/) (2023)
- [VizWiz Grand Challenge: Answering Visual Questions from Blind People](https://mlsanity.com/paper/1802.08218/) (2018)
- [VideoChat: Chat-Centric Video Understanding](https://mlsanity.com/paper/2305.06355/) (2023)

## Built on by (most cited first, 30 of 30)

- [StreamBridge: Turning Your Offline Video Large Language Model into a Proactive Streaming Assistant](https://mlsanity.com/paper/2505.05467/) (2025)
- [InfiniPot-V: Memory-Constrained KV Cache Compression for Streaming Video Understanding](https://mlsanity.com/paper/2506.15745/) (2025)
- [UniGen: Enhanced Training & Test-Time Strategies for Unified Multimodal Understanding and Generation](https://mlsanity.com/paper/2505.14682/) (2025)
- [AToken: A Unified Tokenizer for Vision](https://mlsanity.com/paper/2509.14476/) (2025)
- [A Survey of Token Compression for Efficient Multimodal Large Language Models](https://mlsanity.com/paper/2507.20198/) (2025)
- [FlexSelect: Flexible Token Selection for Efficient Long Video Understanding](https://mlsanity.com/paper/2506.00993/) (2025)
- [VideoSeek: Long-Horizon Video Agent with Tool-Guided Seeking](https://mlsanity.com/paper/2603.20185/) (2026)
- [From Frames to Clips: Training-free Adaptive Key Clip Selection for Long-Form Video Understanding](https://mlsanity.com/paper/2510.02262/) (2025)
- [UniGen-1.5: Enhancing Image Generation and Editing through Reward Unification in Reinforcement Learning](https://mlsanity.com/paper/2511.14760/) (2025)
- [IPIBench: Evaluating Interactive Proactive Intelligence of MLLMs under Continuous Streams](https://mlsanity.com/paper/2605.27074/) (2026)
- [Spatio-Temporal LLM: Reasoning about Environments and Actions](https://mlsanity.com/paper/2507.05258/) (2025)
- [SeViCES: Unifying Semantic-Visual Evidence Consensus for Long Video Understanding](https://mlsanity.com/paper/2510.20622/) (2025)
- [GCAgent: Long-Video Understanding via Schematic and Narrative Episodic Memory](https://mlsanity.com/paper/2511.12027/) (2025)
- [Stateful Token Reduction for Long-Video Hybrid VLMs](https://mlsanity.com/paper/2603.00198/) (2026)
- [TIR-Flow: Active Video Search and Reasoning with Frozen VLMs](https://mlsanity.com/paper/2601.06176/) (2026)
- [Query-Conditioned Evidential Keyframe Sampling for MLLM-Based Long-Form Video Understanding](https://mlsanity.com/paper/2604.01002/) (2026)
- [Swift Sampling: Selecting Temporal Surprises via Taylor Series](https://mlsanity.com/paper/2605.22678/) (2026)
- [LENS: Adaptive Spatio-Temporal Zooming for Keyframe Sampling in Long-Form Videos](https://mlsanity.com/paper/2607.25125/) (2026)
- [Context-Aware Multi-Turn Visual-Textual Reasoning in LVLMs via Dynamic Memory and Adaptive Visual Guidance](https://mlsanity.com/paper/2509.05669/) (2025)
- [MVP: Enhancing Video Large Language Models via Self-supervised Masked Video Prediction](https://mlsanity.com/paper/2601.03781/) (2026)
- [Dynamic Token Compression for Efficient Video Understanding through Reinforcement Learning](https://mlsanity.com/paper/2603.26365/) (2026)
- [Think, Act, Build: An Agentic Framework with Vision Language Models for Zero-Shot 3D Visual Grounding](https://mlsanity.com/paper/2604.00528/) (2026)
- [Towards Temporal Compositional Reasoning in Long-Form Sports Videos](https://mlsanity.com/paper/2604.22226/) (2026)
- [LookWhen? Fast Video Recognition by Learning When, Where, and What to Compute](https://mlsanity.com/paper/2605.06809/) (2026)
- [STORM: Internalized Modeling for Spatial-Temporal Reasoning in Video-Language Models](https://mlsanity.com/paper/2605.26014/) (2026)
- [Learning Geometric Representations from Videos for Spatial Intelligent Multimodal Large Language Models](https://mlsanity.com/paper/2606.05833/) (2026)
- [APB-V: Accelerating Long-Video Understanding via Sequence-Parallelism-aware Approximate Attention](https://mlsanity.com/paper/2601.21444/) (2026)
- [VidPrism: Heterogeneous Mixture of Experts for Image-to-Video Transfer](https://mlsanity.com/paper/2605.28229/) (2026)
- [VisualRouter: Query-Grounded Visual Sampling for Long Video Understanding](https://mlsanity.com/paper/2607.28463/) (2026)
- [VideoResearcher: Self-Improving Tool Design for Long-Video Understanding](https://mlsanity.com/paper/2609.19664/) (2026)

## Abstract (arXiv)

We introduce SlowFast-LLaVA-1.5 (abbreviated as SF-LLaVA-1.5), a family of video large language models (LLMs) offering a token-efficient solution for long-form video understanding. We incorporate the two-stream SlowFast mechanism into a streamlined training pipeline, and perform joint video-image training on a carefully curated data mixture of only publicly available datasets. Our primary focus is on highly efficient model scales (1B and 3B), demonstrating that even relatively small Video LLMs can achieve state-of-the-art performance on video understanding, meeting the demand for mobile-friendly models. Experimental results demonstrate that SF-LLaVA-1.5 achieves superior performance on a wide range of video and image tasks, with robust results at all model sizes (ranging from 1B to 7B). Notably, SF-LLaVA-1.5 achieves state-of-the-art results in long-form video understanding (e.g., LongVideoBench and MLVU) and excels at small scales across various video benchmarks.