EMCompress: Video-LLMs with Endomorphic Multimodal Compression
Not on the map. Why?
Abstract (arXiv)
Video-LLMs face a fundamental tension in long-video reasoning: static, sparse frame sampling either dilutes evidence across task-irrelevant segments at significant cost or misses fine-grained temporal semantics altogether. We propose a novel, cognitively-inspired task -- Endomorphic Multimodal Compression (EMC) -- as a structurally-constrained sufficient-statistic problem for VideoQA, and formulate it as an endomorphic transformation F_EMC : (V, Q) -> (v, q) that compresses the multimodal input while preserving answer invariance across reasonable downstream models. The endomorphic form keeps the compressed output in the downstream pipeline's native task space -- a structural mirror of the filter-then-reason mechanism in the cognitive literature motivating EMC -- distinguishing it from latent-code compression (IB / VIB) and making the formulation extensible to other multimodal settings. Under the Markov chain A -> (V, Q) -> (v, q), EMC realizes the classical sufficiency condition I((v, q); A) = I((V, Q); A) in its VideoQA-natural form. As a modular front-end, EMC plugs into both Video Instruction Tuning and Video Question Answering pipelines. We release the first dedicated benchmark and propose ReSimplifyIt, an EMC baseline surpassing prior methods by 0.40 F-1 with competitive query rewriting. Integrating EMC yields relative gains of 7.33% in training and 33.7% in inference for video-language understanding.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 31 of its 94 references that are in the map. First posted on arXiv 27 Aug 2025.
Builds on
31 of 94 references.
References · 31
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
- 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,620
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
- 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,402
- 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,109
- 2016Deep Variational Information Bottleneckcited by 930
- 2015Deep Learning and the Information Bottleneck Principlecited by 802
- 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 744
- 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 680
- 2017Localizing Moments in Video with Natural Languagecited by 663
19 more
- 2017Towards Automatic Learning of Procedures from Web Instructional Videoscited by 619
- 2017TALL: Temporal Activity Localization via Language Querycited by 584
- 2019ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answeringcited by 561
- 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 482
- 2018TVQA: Localized, Compositional Video Question Answeringcited by 467
- 2024LVBench: An Extreme Long Video Understanding Benchmarkcited by 410
- 2023Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understandingcited by 371
- 2024VideoAgent: Long-form Video Understanding with Large Language Model as Agentcited by 332
- 2024PLLaVA : Parameter-free LLaVA Extension from Images to Videos for Video Dense Captioningcited by 322
- 2024VideoTree: Adaptive Tree-based Video Representation for LLM Reasoning on Long Videoscited by 251
- 2020Span-based Localizing Network for Natural Language Video Localizationcited by 235
- 2018To Find Where You Talk: Temporal Sentence Localization in Video with Attention Based Location Regressioncited by 157
- 2019TVQA+: Spatio-Temporal Grounding for Video Question Answeringcited by 154
- 2018Multilevel Language and Vision Integration for Text-to-Clip Retrievalcited by 128
- 2023Vista-LLaMA: Reducing Hallucination in Video Language Models via Equal Distance to Visual Tokenscited by 71
- 2024Grounded Multi-Hop VideoQA in Long-Form Egocentric Videoscited by 45
- 2023BT-Adapter: Video Conversation is Feasible Without Video Instruction Tuningcited by 41
- 2024Multi-granularity Correspondence Learning from Long-term Noisy Videoscited by 34
- 2024VTG-GPT: Tuning-Free Zero-Shot Video Temporal Grounding with GPTcited by 19
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.