# CAST: Character labeling in Animation using Self-supervision by Tracking

arXiv: https://arxiv.org/abs/2201.07619 · HTML: https://mlsanity.com/paper/2201.07619/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 58 references that are in the map.
- First posted on arXiv 19 Jan 2022.

## Builds on (most cited first, 16 of 16)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [YOLO9000: Better, Faster, Stronger](https://mlsanity.com/paper/1612.08242/) (2016)
- [Towards Universal Object Detection by Domain Attention](https://mlsanity.com/paper/1904.04402/) (2019)
- [Discovering Visual Patterns in Art Collections with Spatially-consistent Feature Learning](https://mlsanity.com/paper/1903.02678/) (2019)
- [Object Detection for Comics using Manga109 Annotations](https://mlsanity.com/paper/1803.08670/) (2018)
- [Rekall: Specifying Video Events using Compositions of Spatiotemporal Labels](https://mlsanity.com/paper/1910.02993/) (2019)
- [Self-Supervised Learning of Face Representations for Video Face Clustering](https://mlsanity.com/paper/1903.01000/) (2019)
- [Human-Centered Emotion Recognition in Animated GIFs](https://mlsanity.com/paper/1904.12201/) (2019)
- [Speaker Naming in Movies](https://mlsanity.com/paper/1809.08761/) (2018)
- [Robust Character Labeling in Movie Videos: Data Resources and Self-supervised Feature Adaptation](https://mlsanity.com/paper/2008.11289/) (2020)
- [Who's that Actor? Automatic Labelling of Actors in TV series starting from IMDB Images](https://mlsanity.com/paper/1611.09162/) (2016)
- [Bringing Cartoons to Life: Towards Improved Cartoon Face Detection and Recognition Systems](https://mlsanity.com/paper/1804.01753/) (2018)

## Built on by (most cited first, 4 of 4)

- [Identity-Aware Semi-Supervised Learning for Comic Character Re-Identification](https://mlsanity.com/paper/2308.09096/) (2023)
- [One missing piece in Vision and Language: A Survey on Comics Understanding](https://mlsanity.com/paper/2409.09502/) (2024)
- [Re:Draw -- Context Aware Translation as a Controllable Method for Artistic Production](https://mlsanity.com/paper/2401.03499/) (2024)
- [Character-Centric Understanding of Animated Movies](https://mlsanity.com/paper/2509.12204/) (2025)

## Abstract (arXiv)

Cartoons and animation domain videos have very different characteristics compared to real-life images and videos. In addition, this domain carries a large variability in styles. Current computer vision and deep-learning solutions often fail on animated content because they were trained on natural images. In this paper we present a method to refine a semantic representation suitable for specific animated content. We first train a neural network on a large-scale set of animation videos and use the mapping to deep features as an embedding space. Next, we use self-supervision to refine the representation for any specific animation style by gathering many examples of animated characters in this style, using a multi-object tracking. These examples are used to define triplets for contrastive loss training. The refined semantic space allows better clustering of animated characters even when they have diverse manifestations. Using this space we can build dictionaries of characters in an animation videos, and define specialized classifiers for specific stylistic content (e.g., characters in a specific animation series) with very little user effort. These classifiers are the basis for automatically labeling characters in animation videos. We present results on a collection of characters in a variety of animation styles.