# Speaker Naming in Movies

arXiv: https://arxiv.org/abs/1809.08761 · HTML: https://mlsanity.com/paper/1809.08761/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Sep 2018.

## Built on by (most cited first, 3 of 3)

- [CAST: Character labeling in Animation using Self-supervision by Tracking](https://mlsanity.com/paper/2201.07619/) (2022)
- [Creating Multimedia Summaries Using Tweets and Videos](https://mlsanity.com/paper/2203.08931/) (2022)
- [VANPY: Voice Analysis Framework](https://mlsanity.com/paper/2502.17579/) (2025)

## Abstract (arXiv)

We propose a new model for speaker naming in movies that leverages visual, textual, and acoustic modalities in an unified optimization framework. To evaluate the performance of our model, we introduce a new dataset consisting of six episodes of the Big Bang Theory TV show and eighteen full movies covering different genres. Our experiments show that our multimodal model significantly outperforms several competitive baselines on the average weighted F-score metric. To demonstrate the effectiveness of our framework, we design an end-to-end memory network model that leverages our speaker naming model and achieves state-of-the-art results on the subtitles task of the MovieQA 2017 Challenge.