# Deep Architectures and Ensembles for Semantic Video Classification

arXiv: https://arxiv.org/abs/1807.01026 · HTML: https://mlsanity.com/paper/1807.01026/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Jul 2018.

## Built on by (most cited first, 6 of 6)

- [Patch-NetVLAD: Multi-Scale Fusion of Locally-Global Descriptors for Place Recognition](https://mlsanity.com/paper/2103.01486/) (2021)
- [Self-Supervised Learning for Place Representation Generalization across Appearance Changes](https://mlsanity.com/paper/2303.02370/) (2023)
- [Attend-Fusion: Efficient Audio-Visual Fusion for Video Classification](https://mlsanity.com/paper/2408.14441/) (2024)
- [Cross-modal Place Recognition in Image Databases using Event-based Sensors](https://mlsanity.com/paper/2307.01047/) (2023)
- [Improving Generalization of Synthetically Trained Sonar Image Descriptors for Underwater Place Recognition](https://mlsanity.com/paper/2308.01058/) (2023)
- [GRM: Gradient Rectification Module for Visual Place Retrieval](https://mlsanity.com/paper/2204.10972/) (2022)

## Abstract (arXiv)

This work addresses the problem of accurate semantic labelling of short videos. To this end, a multitude of different deep nets, ranging from traditional recurrent neural networks (LSTM, GRU), temporal agnostic networks (FV,VLAD,BoW), fully connected neural networks mid-stage AV fusion and others. Additionally, we also propose a residual architecture-based DNN for video classification, with state-of-the art classification performance at significantly reduced complexity. Furthermore, we propose four new approaches to diversity-driven multi-net ensembling, one based on fast correlation measure and three incorporating a DNN-based combiner. We show that significant performance gains can be achieved by ensembling diverse nets and we investigate factors contributing to high diversity. Based on the extensive YouTube8M dataset, we provide an in-depth evaluation and analysis of their behaviour. We show that the performance of the ensemble is state-of-the-art achieving the highest accuracy on the YouTube-8M Kaggle test data. The performance of the ensemble of classifiers was also evaluated on the HMDB51 and UCF101 datasets, and show that the resulting method achieves comparable accuracy with state-of-the-art methods using similar input features.