# Test-Time Adaptation for Video Highlight Detection Using Meta-Auxiliary Learning and Cross-Modality Hallucinations

arXiv: https://arxiv.org/abs/2508.04924 · HTML: https://mlsanity.com/paper/2508.04924/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 47 references that are in the map.
- First posted on arXiv 6 Aug 2025.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [Unsupervised Data Augmentation for Consistency Training](https://mlsanity.com/paper/1904.12848/) (2019)
- [PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition](https://mlsanity.com/paper/1912.10211/) (2019)
- [Efficient Test-Time Model Adaptation without Forgetting](https://mlsanity.com/paper/2204.02610/) (2022)
- [UniVTG: Towards Unified Video-Language Temporal Grounding](https://mlsanity.com/paper/2307.16715/) (2023)
- [Query-Dependent Video Representation for Moment Retrieval and Highlight Detection](https://mlsanity.com/paper/2303.13874/) (2023)
- [UMT: Unified Multi-modal Transformers for Joint Video Moment Retrieval and Highlight Detection](https://mlsanity.com/paper/2203.12745/) (2022)
- [Entropy is not Enough for Test-Time Adaptation: From the Perspective of Disentangled Factors](https://mlsanity.com/paper/2403.07366/) (2024)
- [Self-Supervised Generalisation with Meta Auxiliary Learning](https://mlsanity.com/paper/1901.08933/) (2019)
- [Debiased Learning from Naturally Imbalanced Pseudo-Labels](https://mlsanity.com/paper/2201.01490/) (2022)
- [Less is More: Learning Highlight Detection from Video Duration](https://mlsanity.com/paper/1903.00859/) (2019)
- [TR-DETR: Task-Reciprocal Transformer for Joint Moment Retrieval and Highlight Detection](https://mlsanity.com/paper/2401.02309/) (2024)
- [MINI-Net: Multiple Instance Ranking Network for Video Highlight Detection](https://mlsanity.com/paper/2007.09833/) (2020)
- [Video2GIF: Automatic Generation of Animated GIFs from Video](https://mlsanity.com/paper/1605.04850/) (2016)
- [Learning Trailer Moments in Full-Length Movies](https://mlsanity.com/paper/2008.08502/) (2020)
- [Video Summarization by Learning from Unpaired Data](https://mlsanity.com/paper/1805.12174/) (2018)
- [To Click or Not To Click: Automatic Selection of Beautiful Thumbnails from Videos](https://mlsanity.com/paper/1609.01388/) (2016)
- [Unsupervised Extraction of Video Highlights Via Robust Recurrent Auto-encoders](https://mlsanity.com/paper/1510.01442/) (2015)
- [Task-Driven Exploration: Decoupling and Inter-Task Feedback for Joint Moment Retrieval and Highlight Detection](https://mlsanity.com/paper/2404.09263/) (2024)
- [Point-TTA: Test-Time Adaptation for Point Cloud Registration Using Multitask Meta-Auxiliary Learning](https://mlsanity.com/paper/2308.16481/) (2023)
- [Adaptive Video Highlight Detection by Learning from User History](https://mlsanity.com/paper/2007.09598/) (2020)
- [A Deep Ranking Model for Spatio-Temporal Highlight Detection from a 360 Video](https://mlsanity.com/paper/1801.10312/) (2018)
- [ProtoCon: Pseudo-label Refinement via Online Clustering and Prototypical Consistency for Efficient Semi-supervised Learning](https://mlsanity.com/paper/2303.13556/) (2023)
- [Test-Time Adaptation for Point Cloud Upsampling Using Meta-Learning](https://mlsanity.com/paper/2308.16484/) (2023)
- [Unsupervised Modality-Transferable Video Highlight Detection with Representation Activation Sequence Learning](https://mlsanity.com/paper/2403.09401/) (2024)
- [Unsupervised Video Highlight Detection by Learning from Audio and Visual Recurrence](https://mlsanity.com/paper/2407.13933/) (2024)

## Built on by (most cited first, 2 of 2)

- [SGWIB:Sliced Gromov-Wasserstein Information Bottleneck for Video Highlight Detection](https://mlsanity.com/paper/2609.13966/) (2026)
- [Audio for Sports Highlight Detection: A Comparative Empirical Study](https://mlsanity.com/paper/2609.17923/) (2026)

## Abstract (arXiv)

Existing video highlight detection methods, although advanced, struggle to generalize well to all test videos. These methods typically employ a generic highlight detection model for each test video, which is suboptimal as it fails to account for the unique characteristics and variations of individual test videos. Such fixed models do not adapt to the diverse content, styles, or audio and visual qualities present in new, unseen test videos, leading to reduced highlight detection performance. In this paper, we propose Highlight-TTA, a test-time adaptation framework for video highlight detection that addresses this limitation by dynamically adapting the model during testing to better align with the specific characteristics of each test video, thereby improving generalization and highlight detection performance. Highlight-TTA is jointly optimized with an auxiliary task, cross-modality hallucinations, alongside the primary highlight detection task. We utilize a meta-auxiliary training scheme to enable effective adaptation through the auxiliary task while enhancing the primary task. During testing, we adapt the trained model using the auxiliary task on the test video to further enhance its highlight detection performance. Extensive experiments with three state-of-the-art highlight detection models and three benchmark datasets show that the introduction of Highlight-TTA to these models improves their performance, yielding superior results.