# AntPivot: Livestream Highlight Detection via Hierarchical Attention Mechanism

arXiv: https://arxiv.org/abs/2206.04888 · HTML: https://mlsanity.com/paper/2206.04888/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 37 references that are in the map.
- First posted on arXiv 10 Jun 2022.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Video Summarization with Long Short-term Memory](https://mlsanity.com/paper/1605.08110/) (2016)
- [Relaxed Transformer Decoders for Direct Action Proposal Generation](https://mlsanity.com/paper/2102.01894/) (2021)
- [TURN TAP: Temporal Unit Regression Network for Temporal Action Proposals](https://mlsanity.com/paper/1703.06189/) (2017)
- [BSN++: Complementary Boundary Regressor with Scale-Balanced Relation Modeling for Temporal Action Proposal Generation](https://mlsanity.com/paper/2009.07641/) (2020)
- [A Local-to-Global Approach to Multi-modal Movie Scene Segmentation](https://mlsanity.com/paper/2004.02678/) (2020)
- [Text Segmentation by Cross Segment Attention](https://mlsanity.com/paper/2004.14535/) (2020)
- [A Deep Siamese Network for Scene Detection in Broadcast Videos](https://mlsanity.com/paper/1510.08893/) (2015)
- [Toward Fast and Accurate Neural Discourse Segmentation](https://mlsanity.com/paper/1808.09147/) (2018)
- [Temporal Action Proposal Generation with Background Constraint](https://mlsanity.com/paper/2112.07984/) (2021)
- [StreamHover: Livestream Transcript Summarization and Annotation](https://mlsanity.com/paper/2109.05160/) (2021)
- [Video Summarization via Actionness Ranking](https://mlsanity.com/paper/1903.00110/) (2019)

## Built on by (most cited first, 2 of 2)

- [Temporal Sentence Grounding in Streaming Videos](https://mlsanity.com/paper/2308.07102/) (2023)
- [A Multimodal Transformer for Live Streaming Highlight Prediction](https://mlsanity.com/paper/2407.12002/) (2024)

## Abstract (arXiv)

In recent days, streaming technology has greatly promoted the development in the field of livestream. Due to the excessive length of livestream records, it's quite essential to extract highlight segments with the aim of effective reproduction and redistribution. Although there are lots of approaches proven to be effective in the highlight detection for other modals, the challenges existing in livestream processing, such as the extreme durations, large topic shifts, much irrelevant information and so forth, heavily hamper the adaptation and compatibility of these methods. In this paper, we formulate a new task Livestream Highlight Detection, discuss and analyze the difficulties listed above and propose a novel architecture AntPivot to solve this problem. Concretely, we first encode the original data into multiple views and model their temporal relations to capture clues in a hierarchical attention mechanism. Afterwards, we try to convert the detection of highlight clips into the search for optimal decision sequences and use the fully integrated representations to predict the final results in a dynamic-programming mechanism. Furthermore, we construct a fully-annotated dataset AntHighlight to instantiate this task and evaluate the performance of our model. The extensive experiments indicate the effectiveness and validity of our proposed method.