# From Benedict Cumberbatch to Sherlock Holmes: Character Identification in TV series without a Script

arXiv: https://arxiv.org/abs/1801.10442 · HTML: https://mlsanity.com/paper/1801.10442/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Jan 2018.

## Built on by (most cited first, 20 of 20)

- [MM-VID: Advancing Video Understanding with GPT-4V(ision)](https://mlsanity.com/paper/2310.19773/) (2023)
- [AutoAD II: The Sequel -- Who, When, and What in Movie Audio Description](https://mlsanity.com/paper/2310.06838/) (2023)
- [The One Where They Reconstructed 3D Humans and Environments in TV Shows](https://mlsanity.com/paper/2207.14279/) (2022)
- [MovieCuts: A New Dataset and Benchmark for Cut Type Recognition](https://mlsanity.com/paper/2109.05569/) (2021)
- [Face, Body, Voice: Video Person-Clustering with Multiple Modalities](https://mlsanity.com/paper/2105.09939/) (2021)
- [OPOM: Customized Invisible Cloak towards Face Privacy Protection](https://mlsanity.com/paper/2205.11981/) (2022)
- [StoryTeller: Improving Long Video Description through Global Audio-Visual Character Identification](https://mlsanity.com/paper/2411.07076/) (2024)
- [Tails Tell Tales: Chapter-Wide Manga Transcriptions with Character Names](https://mlsanity.com/paper/2408.00298/) (2024)
- [Automated Video Labelling: Identifying Faces by Corroborative Evidence](https://mlsanity.com/paper/2102.05645/) (2021)
- [MICap: A Unified Model for Identity-aware Movie Descriptions](https://mlsanity.com/paper/2405.11483/) (2024)
- [Look, Listen and Recognise: Character-Aware Audio-Visual Subtitling](https://mlsanity.com/paper/2401.12039/) (2024)
- [How you feelin'? Learning Emotions and Mental States in Movie Scenes](https://mlsanity.com/paper/2304.05634/) (2023)
- [SynopGround: A Large-Scale Dataset for Multi-Paragraph Video Grounding from TV Dramas and Synopses](https://mlsanity.com/paper/2408.01669/) (2024)
- [APES: Audiovisual Person Search in Untrimmed Video](https://mlsanity.com/paper/2106.01667/) (2021)
- [DANTE-AD: Dual-Vision Attention Network for Long-Term Audio Description](https://mlsanity.com/paper/2503.24096/) (2025)
- [FocusedAD: Character-centric Movie Audio Description](https://mlsanity.com/paper/2504.12157/) (2025)
- [Character-aware audio-visual subtitling in context](https://mlsanity.com/paper/2410.11068/) (2024)
- ["Previously on ..." From Recaps to Story Summarization](https://mlsanity.com/paper/2405.11487/) (2024)
- [Creating Multimedia Summaries Using Tweets and Videos](https://mlsanity.com/paper/2203.08931/) (2022)
- [Beyond Anonymous Captions: Grounding Character Identity in Video Captioning and Question Answering](https://mlsanity.com/paper/2610.10163/) (2026)

## Abstract (arXiv)

The goal of this paper is the automatic identification of characters in TV and feature film material. In contrast to standard approaches to this task, which rely on the weak supervision afforded by transcripts and subtitles, we propose a new method requiring only a cast list. This list is used to obtain images of actors from freely available sources on the web, providing a form of partial supervision for this task. In using images of actors to recognize characters, we make the following three contributions: (i) We demonstrate that an automated semi-supervised learning approach is able to adapt from the actor's face to the character's face, including the face context of the hair; (ii) By building voice models for every character, we provide a bridge between frontal faces (for which there is plenty of actor-level supervision) and profile (for which there is very little or none); and (iii) by combining face context and speaker identification, we are able to identify characters with partially occluded faces and extreme facial poses. Results are presented on the TV series 'Sherlock' and the feature film 'Casablanca'. We achieve the state-of-the-art on the Casablanca benchmark, surpassing previous methods that have used the stronger supervision available from transcripts.