mlsanityData to 14 Oct 2026

EMCompress: Video-LLMs with Endomorphic Multimodal Compression

arXiv 2508.21094 ↗First posted 27 Aug 2025cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

Video-LLMs face a fundamental tension in long-video reasoning: static, sparse frame sampling either dilutes evidence across task-irrelevant segments at significant cost or misses fine-grained temporal semantics altogether. We propose a novel, cognitively-inspired task -- Endomorphic Multimodal Compression (EMC) -- as a structurally-constrained sufficient-statistic problem for VideoQA, and formulate it as an endomorphic transformation F_EMC : (V, Q) -> (v, q) that compresses the multimodal input while preserving answer invariance across reasonable downstream models. The endomorphic form keeps the compressed output in the downstream pipeline's native task space -- a structural mirror of the filter-then-reason mechanism in the cognitive literature motivating EMC -- distinguishing it from latent-code compression (IB / VIB) and making the formulation extensible to other multimodal settings. Under the Markov chain A -> (V, Q) -> (v, q), EMC realizes the classical sufficiency condition I((v, q); A) = I((V, Q); A) in its VideoQA-natural form. As a modular front-end, EMC plugs into both Video Instruction Tuning and Video Question Answering pipelines. We release the first dedicated benchmark and propose ReSimplifyIt, an EMC baseline surpassing prior methods by 0.40 F-1 with competitive query rewriting. Integrating EMC yields relative gains of 7.33% in training and 33.7% in inference for video-language understanding.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 31 of its 94 references that are in the map. First posted on arXiv 27 Aug 2025.

Builds on

31 of 94 references.

References · 31

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  3. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  4. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,620
  5. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
  6. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,402
  7. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,109
  8. 2016Deep Variational Information Bottleneckcited by 930
  9. 2015Deep Learning and the Information Bottleneck Principlecited by 802
  10. 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 744
  11. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 680
  12. 2017Localizing Moments in Video with Natural Languagecited by 663
19 more
  1. 2017Towards Automatic Learning of Procedures from Web Instructional Videoscited by 619
  2. 2017TALL: Temporal Activity Localization via Language Querycited by 584
  3. 2019ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answeringcited by 561
  4. 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 482
  5. 2018TVQA: Localized, Compositional Video Question Answeringcited by 467
  6. 2024LVBench: An Extreme Long Video Understanding Benchmarkcited by 410
  7. 2023Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understandingcited by 371
  8. 2024VideoAgent: Long-form Video Understanding with Large Language Model as Agentcited by 332
  9. 2024PLLaVA : Parameter-free LLaVA Extension from Images to Videos for Video Dense Captioningcited by 322
  10. 2024VideoTree: Adaptive Tree-based Video Representation for LLM Reasoning on Long Videoscited by 251
  11. 2020Span-based Localizing Network for Natural Language Video Localizationcited by 235
  12. 2018To Find Where You Talk: Temporal Sentence Localization in Video with Attention Based Location Regressioncited by 157
  13. 2019TVQA+: Spatio-Temporal Grounding for Video Question Answeringcited by 154
  14. 2018Multilevel Language and Vision Integration for Text-to-Clip Retrievalcited by 128
  15. 2023Vista-LLaMA: Reducing Hallucination in Video Language Models via Equal Distance to Visual Tokenscited by 71
  16. 2024Grounded Multi-Hop VideoQA in Long-Form Egocentric Videoscited by 45
  17. 2023BT-Adapter: Video Conversation is Feasible Without Video Instruction Tuningcited by 41
  18. 2024Multi-granularity Correspondence Learning from Long-term Noisy Videoscited by 34
  19. 2024VTG-GPT: Tuning-Free Zero-Shot Video Temporal Grounding with GPTcited by 19

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.