# VIBE: Annotation-Free Video-to-Text Information Bottleneck Evaluation for TL;DR

arXiv: https://arxiv.org/abs/2505.17423 · HTML: https://mlsanity.com/paper/2505.17423/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 50 references that are in the map.
- First posted on arXiv 23 May 2025.

## Builds on (most cited first, 27 of 27)

- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [CIDEr: Consensus-based Image Description Evaluation](https://mlsanity.com/paper/1411.5726/) (2014)
- [InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models](https://mlsanity.com/paper/2504.10479/) (2025)
- [Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling](https://mlsanity.com/paper/2412.05271/) (2024)
- [Deep Learning and the Information Bottleneck Principle](https://mlsanity.com/paper/1503.02406/) (2015)
- [RouteLLM: Learning to Route LLMs with Preference Data](https://mlsanity.com/paper/2406.18665/) (2024)
- [LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding](https://mlsanity.com/paper/2407.15754/) (2024)
- [Mixture-of-Experts with Expert Choice Routing](https://mlsanity.com/paper/2202.09368/) (2022)
- [VATEX: A Large-Scale, High-Quality Multilingual Dataset for Video-and-Language Research](https://mlsanity.com/paper/1904.03493/) (2019)
- [AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmark](https://mlsanity.com/paper/2410.03051/) (2024)
- [How Does Information Bottleneck Help Deep Learning?](https://mlsanity.com/paper/2305.18887/) (2023)
- [SUTD-TrafficQA: A Question Answering Benchmark and an Efficient Network for Video Reasoning over Traffic Events](https://mlsanity.com/paper/2103.15538/) (2021)
- [Video Question Answering: Datasets, Algorithms and Challenges](https://mlsanity.com/paper/2203.01225/) (2022)
- [The Information Bottleneck Problem and Its Applications in Machine Learning](https://mlsanity.com/paper/2004.14941/) (2020)
- [Spoken Moments: Learning Joint Audio-Visual Representations from Video Descriptions](https://mlsanity.com/paper/2105.04489/) (2021)
- [Multimodal Information Bottleneck: Learning Minimal Sufficient Unimodal and Multimodal Representations](https://mlsanity.com/paper/2210.17444/) (2022)
- [BottleSum: Unsupervised and Self-supervised Sentence Summarization using the Information Bottleneck Principle](https://mlsanity.com/paper/1909.07405/) (2019)
- [Accelerating Human-in-the-loop Machine Learning: Challenges and Opportunities](https://mlsanity.com/paper/1804.05892/) (2018)
- [Improving the Adversarial Robustness of NLP Models by Information Bottleneck](https://mlsanity.com/paper/2206.05511/) (2022)
- [Representation Learning in Deep RL via Discrete Information Bottleneck](https://mlsanity.com/paper/2212.13835/) (2022)
- [Online Foundation Model Selection in Robotics](https://mlsanity.com/paper/2402.08570/) (2024)
- [Is Summary Useful or Not? An Extrinsic Human Evaluation of Text Summaries on Downstream Tasks](https://mlsanity.com/paper/2305.15044/) (2023)
- [Information-Theoretic Distillation for Reference-less Summarization](https://mlsanity.com/paper/2403.13780/) (2024)
- [Human-Agent Cooperation in Games under Incomplete Information through Natural Language Communication](https://mlsanity.com/paper/2405.14173/) (2024)
- [Exploiting Distribution Constraints for Scalable and Efficient Image Retrieval](https://mlsanity.com/paper/2410.07022/) (2024)
- [Any2Any: Incomplete Multimodal Retrieval with Conformal Prediction](https://mlsanity.com/paper/2411.10513/) (2024)

## Built on by (most cited first, 3 of 3)

- [ViSIL: Unified Evaluation of Information Loss in Multimodal Video Captioning](https://mlsanity.com/paper/2601.09851/) (2026)
- [What We are Missing in Multimodal LLM Evaluation?](https://mlsanity.com/paper/2606.26348/) (2026)
- [VEGAS: Human-Aligned Video Caption Evaluation via Gaze](https://mlsanity.com/paper/2607.08489/) (2026)

## Abstract (arXiv)

Many decision-making tasks, where both accuracy and efficiency matter, still require human supervision. For example, tasks like traffic officers reviewing hour-long dashcam footage or researchers screening conference videos can benefit from concise summaries that reduce cognitive load and save time. Yet current vision-language models (VLMs) often produce verbose, redundant outputs that hinder task performance. Existing video caption evaluation depends on costly human annotations and overlooks the summaries' utility in downstream tasks. We address these gaps with Video-to-text Information Bottleneck Evaluation (VIBE), an annotation-free method that scores VLM outputs using two metrics: grounding (how well the summary aligns with visual content) and utility (how informative it is for the task). VIBE selects from randomly sampled VLM outputs by ranking them according to the two scores to support effective human decision-making. Human studies on LearningPaper24, SUTD-TrafficQA, and LongVideoBench show that summaries selected by VIBE consistently improve performance-boosting task accuracy by up to 61.23% and reducing response time by 75.77% compared to naive VLM summaries or raw video.