# Training-Free Temporal Abstraction for General Video Understanding

arXiv: https://arxiv.org/abs/2608.27929 · HTML: https://mlsanity.com/paper/2608.27929/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 42 of its 47 references that are in the map.
- First posted on arXiv 28 Aug 2026.

## Builds on (most cited first, 30 of 42)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Qwen3-VL Technical Report](https://mlsanity.com/paper/2511.21631/) (2025)
- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis](https://mlsanity.com/paper/2405.21075/) (2024)
- [Dense-Captioning Events in Videos](https://mlsanity.com/paper/1705.00754/) (2017)
- [LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding](https://mlsanity.com/paper/2407.15754/) (2024)
- [Fine-tuning CNN Image Retrieval with No Human Annotation](https://mlsanity.com/paper/1711.02512/) (2017)
- [LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Models](https://mlsanity.com/paper/2407.12772/) (2024)
- [BMN: Boundary-Matching Network for Temporal Action Proposal Generation](https://mlsanity.com/paper/1907.09702/) (2019)
- [InternVideo2: Scaling Foundation Models for Multimodal Video Understanding](https://mlsanity.com/paper/2403.15377/) (2024)
- [Learning 2D Temporal Adjacent Networks for Moment Localization with Natural Language](https://mlsanity.com/paper/1912.03590/) (2019)
- [X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval](https://mlsanity.com/paper/2207.07285/) (2022)
- [VideoTree: Adaptive Tree-based Video Representation for LLM Reasoning on Long Videos](https://mlsanity.com/paper/2405.19209/) (2024)
- [UniVTG: Towards Unified Video-Language Temporal Grounding](https://mlsanity.com/paper/2307.16715/) (2023)
- [Adaptive Keyframe Sampling for Long Video Understanding](https://mlsanity.com/paper/2502.21271/) (2025)
- [Query-Dependent Video Representation for Moment Retrieval and Highlight Detection](https://mlsanity.com/paper/2303.13874/) (2023)
- [Semantic Conditioned Dynamic Modulation for Temporal Sentence Grounding in Videos](https://mlsanity.com/paper/1910.14303/) (2019)
- [Frame-Voyager: Learning to Query Frames for Video Large Language Models](https://mlsanity.com/paper/2410.03226/) (2024)
- [Segmental Spatiotemporal CNNs for Fine-grained Action Segmentation](https://mlsanity.com/paper/1602.02995/) (2016)
- [BOLT: Boost Large Vision-Language Model Without Training for Long-form Video Understanding](https://mlsanity.com/paper/2503.21483/) (2025)
- [UnLoc: A Unified Framework for Video Localization Tasks](https://mlsanity.com/paper/2308.11062/) (2023)
- [Generic Event Boundary Detection: A Benchmark for Event Segmentation](https://mlsanity.com/paper/2101.10511/) (2021)
- [TR-DETR: Task-Reciprocal Transformer for Joint Moment Retrieval and Highlight Detection](https://mlsanity.com/paper/2401.02309/) (2024)
- [ChatVTG: Video Temporal Grounding via Chat with Video Dialogue Large Language Models](https://mlsanity.com/paper/2410.12813/) (2024)
- [Intra- and Inter-Action Understanding via Temporal Action Parsing](https://mlsanity.com/paper/2005.10229/) (2020)
- [Training-free Video Temporal Grounding using Large-scale Pre-trained Models](https://mlsanity.com/paper/2408.16219/) (2024)
- [Zero-Shot Video Moment Retrieval from Frozen Vision-Language Models](https://mlsanity.com/paper/2309.00661/) (2023)
- [Saliency-Guided DETR for Moment Retrieval and Highlight Detection](https://mlsanity.com/paper/2410.01615/) (2024)
- [Zero-shot Video Moment Retrieval via Off-the-shelf Multimodal Large Language Models](https://mlsanity.com/paper/2501.07972/) (2025)
- [Temporal Perceiver: A General Architecture for Arbitrary Boundary Detection](https://mlsanity.com/paper/2203.00307/) (2022)

## Abstract (arXiv)

Videos are expensive to analyze frame by frame, yet many video understanding tasks depend on knowing where relevant moments occur. A system may need to find when an action changes, locate the segment described by a sentence, or choose a few frames for a vision-language model. Existing methods often solve these problems separately, using task-specific training data or specialized architectures. We study whether a pretrained video-text model can provide enough temporal structure to support several of these tasks at once. We present STITCH, a training-free method that divides a video into semantically meaningful temporal chunks. STITCH embeds short video windows with a frozen video-text backbone and detects changes in the resulting embedding sequence. These chunks are computed once per video and reused across tasks. We evaluate STITCH on generic event boundary detection, language-based moment retrieval, and frame selection for long-video VLM reasoning. Across all three settings, STITCH remains competitive with more specialized methods while requiring no task-specific training, with especially clear gains when only a small number of frames or tokens can be processed. These results suggest that reusable temporal abstraction is a promising direction for general video understanding, allowing dense video streams to be converted once into semantic units that can be localized, retrieved, sampled, or reasoned over by downstream systems.