# Decoding the Hook: A Multimodal LLM Framework for Analyzing the Hooking Period of Video Ads

arXiv: https://arxiv.org/abs/2602.22299 · HTML: https://mlsanity.com/paper/2602.22299/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 29 references that are in the map.
- First posted on arXiv 25 Feb 2026.

## Builds on (most cited first, 17 of 17)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Is Space-Time Attention All You Need for Video Understanding?](https://mlsanity.com/paper/2102.05095/) (2021)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [Less is More: ClipBERT for Video-and-Language Learning via Sparse Sampling](https://mlsanity.com/paper/2102.06183/) (2021)
- [Multi-modal Transformer for Video Retrieval](https://mlsanity.com/paper/2007.10639/) (2020)
- [Expanding Language-Image Pretrained Models for General Video Recognition](https://mlsanity.com/paper/2208.02816/) (2022)
- [Adaptive Keyframe Sampling for Long Video Understanding](https://mlsanity.com/paper/2502.21271/) (2025)
- [Listen to Look: Action Recognition by Previewing Audio](https://mlsanity.com/paper/1912.04487/) (2019)
- [T*: Re-thinking Temporal Search for Long-Form Video Understanding](https://mlsanity.com/paper/2504.02259/) (2025)
- [Parameter Efficient Multimodal Transformers for Video Representation Learning](https://mlsanity.com/paper/2012.04124/) (2020)
- [Deep CTR Prediction in Display Advertising](https://mlsanity.com/paper/1609.06018/) (2016)
- [User Response Prediction in Online Advertising](https://mlsanity.com/paper/2101.02342/) (2021)
- [LoVR: A Benchmark for Long Video Retrieval in Multimodal Contexts](https://mlsanity.com/paper/2505.13928/) (2025)
- [Ad-load Balancing via Off-policy Learning in a Content Marketplace](https://mlsanity.com/paper/2309.11518/) (2023)
- [Multi-modal Representation Learning for Video Advertisement Content Structuring](https://mlsanity.com/paper/2109.06637/) (2021)
- [Predicting Online Video Advertising Effects with Multimodal Deep Learning](https://mlsanity.com/paper/2012.11851/) (2020)

## Abstract (arXiv)

Video-based ads are a vital medium for brands to engage consumers, with social media platforms leveraging user data to optimize ad delivery and boost engagement. A crucial but under-explored aspect is the 'hooking period', the first three seconds that capture viewer attention and influence engagement metrics. Analyzing this brief window is challenging due to the multimodal nature of video content, which blends visual, auditory, and textual elements. Traditional methods often miss the nuanced interplay of these components, requiring advanced frameworks for thorough evaluation. This study presents a framework using transformer-based multimodal large language models (MLLMs) to analyze the hooking period of video ads. It tests two frame sampling strategies, uniform random sampling and key frame selection, to ensure balanced and representative acoustic feature extraction, capturing the full range of design elements. The hooking video is processed by state-of-the-art MLLMs to generate descriptive analyses of the ad's initial impact, which are distilled into coherent topics using BERTopic for high-level abstraction. The framework also integrates features such as audio attributes and aggregated ad targeting information, enriching the feature set for further analysis. Empirical validation on large-scale real-world data from social media platforms demonstrates the efficacy of our framework, revealing correlations between hooking period features and key performance metrics like conversion per investment. The results highlight the practical applicability and predictive power of the approach, offering valuable insights for optimizing video ad strategies. This study advances video ad analysis by providing a scalable methodology for understanding and enhancing the initial moments of video advertisements.