# VEMOCLAP: A video emotion classification web application

arXiv: https://arxiv.org/abs/2410.21303 · HTML: https://mlsanity.com/paper/2410.21303/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 24 references that are in the map.
- First posted on arXiv 22 Oct 2024.

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [No Language Left Behind: Scaling Human-Centered Machine Translation](https://mlsanity.com/paper/2207.04672/) (2022)
- [ClipCap: CLIP Prefix for Image Captioning](https://mlsanity.com/paper/2111.09734/) (2021)
- [BEATs: Audio Pre-Training with Acoustic Tokenizers](https://mlsanity.com/paper/2212.09058/) (2022)
- [TweetNLP: Cutting-Edge Natural Language Processing for Social Media](https://mlsanity.com/paper/2206.14774/) (2022)
- [An End-to-End Visual-Audio Attention Network for Emotion Recognition in User-Generated Videos](https://mlsanity.com/paper/2003.00832/) (2020)
- [Heterogeneous Knowledge Transfer in Video Emotion Recognition, Attribution and Summarization](https://mlsanity.com/paper/1511.04798/) (2015)
- [Movie Trailer Genre Classification Using Multimodal Pretrained Features](https://mlsanity.com/paper/2410.19760/) (2024)

## Built on by (most cited first, 1 of 1)

- [Video Soundtrack Generation by Aligning Emotions and Temporal Boundaries](https://mlsanity.com/paper/2502.10154/) (2025)

## Abstract (arXiv)

We introduce VEMOCLAP: Video EMOtion Classifier using Pretrained features, the first readily available and open-source web application that analyzes the emotional content of any user-provided video. We improve our previous work, which exploits open-source pretrained models that work on video frames and audio, and then efficiently fuse the resulting pretrained features using multi-head cross-attention. Our approach increases the state-of-the-art classification accuracy on the Ekman-6 video emotion dataset by 4.3% and offers an online application for users to run our model on their own videos or YouTube videos. We invite the readers to try our application at serkansulun.com/app.