# BERT for Large-scale Video Segment Classification with Test-time Augmentation

arXiv: https://arxiv.org/abs/1912.01127 · HTML: https://mlsanity.com/paper/1912.01127/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Dec 2019.

## Built on by (most cited first, 2 of 2)

- [Highlight Timestamp Detection Model for Comedy Videos via Multimodal Sentiment Analysis](https://mlsanity.com/paper/2106.00451/) (2021)
- [Enhancing Video Understanding: Deep Neural Networks for Spatiotemporal Analysis](https://mlsanity.com/paper/2502.07277/) (2025)

## Abstract (arXiv)

This paper presents our approach to the third YouTube-8M video understanding competition that challenges par-ticipants to localize video-level labels at scale to the pre-cise time in the video where the label actually occurs. Ourmodel is an ensemble of frame-level models such as GatedNetVLAD and NeXtVLAD and various BERT models withtest-time augmentation. We explore multiple ways to ag-gregate BERT outputs as video representation and variousways to combine visual and audio information. We proposetest-time augmentation as shifting video frames to one leftor right unit, which adds variety to the predictions and em-pirically shows improvement in evaluation metrics. We firstpre-train the model on the 4M training video-level data, andthen fine-tune the model on 237K annotated video segment-level data. We achieve MAP@100K 0.7871 on private test-ing video segment data, which is ranked 9th over 283 teams.