# CASE: Cost-Aware Stopping for Efficient Long-Video Agents

arXiv: https://arxiv.org/abs/2610.05400 · HTML: https://mlsanity.com/paper/2610.05400/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Agent memory systems, one of 16 directions in LLM agents and tool use.
- Builds on 37 of its 44 references that are in the map.
- First posted on arXiv 4 Oct 2026.

## Placed in

- LLM agents and tool use / [Agent memory systems](https://mlsanity.com/agents/agent-memory-systems/)

## Builds on (most cited first, 30 of 37)

- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis](https://mlsanity.com/paper/2405.21075/) (2024)
- [LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding](https://mlsanity.com/paper/2407.15754/) (2024)
- [Adaptive Computation Time for Recurrent Neural Networks](https://mlsanity.com/paper/1603.08983/) (2016)
- [SelectiveNet: A Deep Neural Network with an Integrated Reject Option](https://mlsanity.com/paper/1901.09192/) (2019)
- [Smart "Predict, then Optimize"](https://mlsanity.com/paper/1710.08005/) (2017)
- [VideoAgent: Long-form Video Understanding with Large Language Model as Agent](https://mlsanity.com/paper/2403.10517/) (2024)
- [Confident Adaptive Language Modeling](https://mlsanity.com/paper/2207.07061/) (2022)
- [MLVU: Benchmarking Multi-task Long Video Understanding](https://mlsanity.com/paper/2406.04264/) (2024)
- [BERT Loses Patience: Fast and Robust Inference with Early Exit](https://mlsanity.com/paper/2006.04152/) (2020)
- [VideoTree: Adaptive Tree-based Video Representation for LLM Reasoning on Long Videos](https://mlsanity.com/paper/2405.19209/) (2024)
- [VideoAgent: A Memory-augmented Multimodal Agent for Video Understanding](https://mlsanity.com/paper/2403.11481/) (2024)
- [PonderNet: Learning to Ponder](https://mlsanity.com/paper/2107.05407/) (2021)
- [Adaptive Keyframe Sampling for Long Video Understanding](https://mlsanity.com/paper/2502.21271/) (2025)
- [Deep Video Discovery: Agentic Search with Tool Use for Long-form Video Understanding](https://mlsanity.com/paper/2505.18079/) (2025)
- [Archon: An Architecture Search Framework for Inference-Time Techniques](https://mlsanity.com/paper/2409.15254/) (2024)
- [STORM: Token-Efficient Long Video Understanding for Multimodal LLMs](https://mlsanity.com/paper/2503.04130/) (2025)
- [AdaReTaKe: Adaptive Redundancy Reduction to Perceive Longer for Video-language Understanding](https://mlsanity.com/paper/2503.12559/) (2025)
- [VideoAgent2: Enhancing the LLM-Based Agent System for Long-Form Video Understanding by Uncertainty-Aware CoT](https://mlsanity.com/paper/2504.04471/) (2025)
- [FOCUS: Efficient Keyframe Selection for Long Video Understanding](https://mlsanity.com/paper/2510.27280/) (2025)
- [Learning to Stop While Learning to Predict](https://mlsanity.com/paper/2006.05082/) (2020)
- [Fast yet Safe: Early-Exiting with Risk Control](https://mlsanity.com/paper/2405.20915/) (2024)
- [Probing-RAG: Self-Probing to Guide Language Models in Selective Document Retrieval](https://mlsanity.com/paper/2410.13339/) (2024)
- [Active Video Perception: Iterative Evidence Seeking for Agentic Long Video Understanding](https://mlsanity.com/paper/2512.05774/) (2025)
- [LongVideoAgent: Multi-Agent Reasoning with Long Videos](https://mlsanity.com/paper/2512.20618/) (2025)
- [TimeSearch-R: Adaptive Temporal Search for Long-Form Video Understanding via Self-Verification Reinforcement Learning](https://mlsanity.com/paper/2511.05489/) (2025)
- [VideoSeek: Long-Horizon Video Agent with Tool-Guided Seeking](https://mlsanity.com/paper/2603.20185/) (2026)
- [Scaling Inference-Efficient Language Models](https://mlsanity.com/paper/2501.18107/) (2025)
- [Stop-RAG: Value-Based Retrieval Control for Iterative RAG](https://mlsanity.com/paper/2510.14337/) (2025)
- [VideoThinker: Building Agentic VideoLLMs with LLM-Guided Tool Reasoning](https://mlsanity.com/paper/2601.15724/) (2026)

## Abstract (arXiv)

Long-video agents can actively gather question-relevant evidence, but they typically leave a central decision implicit: when has the agent seen enough to answer? We propose CASE, a plug-in termination framework that frames this decision as policy-conditioned sequential stopping. At each causal checkpoint, CASE combines an auxiliary multiple-choice assessment of accumulated evidence with the host agent's execution state. From complete native trajectories, we construct a cost-aware target that compares answering now with stopping later along the same search path, accounting jointly for answer correctness and the full cost of continued reasoning. A lightweight Ridge regressor learns this decision gap and produces STOP/CONTINUE decisions. We evaluate three vision-language models with VideoSeek and AVP. On Video-MME, end-to-end accuracy changes by +0.67 percentage points on average while CASE reduces model-token use by 53.63%. The same frozen policies then transfer zero-shot to LongVideoBench and MLVU, with end-to-end accuracy changes of +3.38 and +4.58 points while saving 58.78% and 51.28% of model tokens, respectively. Across all agent-model-benchmark combinations, CASE attains the highest accuracy-efficiency Pareto-frontier coverage among the compared stopping methods (83.3%) at the selected operating points. Online execution preserves this favorable accuracy-efficiency trade-off and additionally reduces measured runtime by 54.1% on average. CASE provides a plug-in termination framework for long-video reasoning agents, enabling them to decide when further evidence acquisition is no longer worthwhile.