# CPST: Comprehension-Preserving Style Transfer for Multi-Modal Narratives

arXiv: https://arxiv.org/abs/2312.08695 · HTML: https://mlsanity.com/paper/2312.08695/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 28 references that are in the map.
- First posted on arXiv 14 Dec 2023.

## Builds on (most cited first, 12 of 12)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [A Neural Algorithm of Artistic Style](https://mlsanity.com/paper/1508.06576/) (2015)
- [Sketch-based Manga Retrieval using Manga109 Dataset](https://mlsanity.com/paper/1510.04389/) (2015)
- [Neural Style Transfer: A Review](https://mlsanity.com/paper/1705.04058/) (2017)
- [Controlling Perceptual Factors in Neural Style Transfer](https://mlsanity.com/paper/1611.07865/) (2016)
- [Fast Patch-based Style Transfer of Arbitrary Style](https://mlsanity.com/paper/1612.04337/) (2016)
- [The Amazing Mysteries of the Gutter: Drawing Inferences Between Panels in Comic Book Narratives](https://mlsanity.com/paper/1611.05118/) (2016)
- [The shape and simplicity biases of adversarially robust ImageNet-trained CNNs](https://mlsanity.com/paper/2006.09373/) (2020)
- [A survey of comics research in computer science](https://mlsanity.com/paper/1804.05490/) (2018)
- [Son of Zorn's Lemma: Targeted Style Transfer Using Instance-aware Semantic Segmentation](https://mlsanity.com/paper/1701.02357/) (2017)

## Abstract (arXiv)

We investigate the challenges of style transfer in multi-modal visual narratives. Among static visual narratives such as comics and manga, there are distinct visual styles in terms of presentation. They include style features across multiple dimensions, such as panel layout, size, shape, and color. They include both visual and text media elements. The layout of both text and media elements is also significant in terms of narrative communication. The sequential transitions between panels are where readers make inferences about the narrative world. These feature differences provide an interesting challenge for style transfer in which there are distinctions between the processing of features for each modality. We introduce the notion of comprehension-preserving style transfer (CPST) in such multi-modal domains. CPST requires not only traditional metrics of style transfer but also metrics of narrative comprehension. To spur further research in this area, we present an annotated dataset of comics and manga and an initial set of algorithms that utilize separate style transfer modules for the visual, textual, and layout parameters. To test whether the style transfer preserves narrative semantics, we evaluate this algorithm through visual story cloze tests inspired by work in computational cognition of narrative systems. Understanding the connection between style and narrative semantics provides insight for applications ranging from informational brochure designs to data storytelling.