# The Learnable Typewriter: A Generative Approach to Text Analysis

arXiv: https://arxiv.org/abs/2302.01660 · HTML: https://mlsanity.com/paper/2302.01660/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 55 references that are in the map.
- First posted on arXiv 3 Feb 2023.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [MONet: Unsupervised Scene Decomposition and Representation](https://mlsanity.com/paper/1901.11390/) (2019)
- [Multi-Object Representation Learning with Iterative Variational Inference](https://mlsanity.com/paper/1903.00450/) (2019)
- [TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models](https://mlsanity.com/paper/2109.10282/) (2021)
- [Attend, Infer, Repeat: Fast Scene Understanding with Generative Models](https://mlsanity.com/paper/1603.08575/) (2016)
- [Neural Expectation Maximization](https://mlsanity.com/paper/1708.03498/) (2017)
- [ClevrTex: A Texture-Rich Benchmark for Unsupervised Multi-Object Segmentation](https://mlsanity.com/paper/2111.10265/) (2021)
- [Pay Attention to What You Read: Non-recurrent Handwritten Text-Line Recognition](https://mlsanity.com/paper/2005.13044/) (2020)
- [ScrabbleGAN: Semi-Supervised Varying Length Handwritten Text Generation](https://mlsanity.com/paper/2003.10557/) (2020)
- [Deformable Sprites for Unsupervised Video Decomposition](https://mlsanity.com/paper/2204.07151/) (2022)
- [Generative Neurosymbolic Machines](https://mlsanity.com/paper/2010.12152/) (2020)
- [Unsupervised Layered Image Decomposition into Object Prototypes](https://mlsanity.com/paper/2104.14575/) (2021)
- [Efficient Iterative Amortized Inference for Learning Symmetric and Disentangled Multi-Object Representations](https://mlsanity.com/paper/2106.03630/) (2021)
- [MarioNette: Self-Supervised Sprite Learning](https://mlsanity.com/paper/2104.14553/) (2021)
- [Deep Transformation-Invariant Clustering](https://mlsanity.com/paper/2006.11132/) (2020)
- [Learning to Manipulate Individual Objects in an Image](https://mlsanity.com/paper/2004.05495/) (2020)
- [docExtractor: An off-the-shelf historical document element extraction](https://mlsanity.com/paper/2012.08191/) (2020)
- [A Generalization of Otsu's Method and Minimum Error Thresholding](https://mlsanity.com/paper/2007.07350/) (2020)
- [A Few-shot Learning Approach for Historical Ciphered Manuscript Recognition](https://mlsanity.com/paper/2009.12577/) (2020)
- [Adaptive Text Recognition through Visual Matching](https://mlsanity.com/paper/2009.06610/) (2020)
- [Learning to Read by Spelling: Towards Unsupervised Text Recognition](https://mlsanity.com/paper/1809.08675/) (2018)
- [Search for Concepts: Discovering Visual Concepts Using Direct Optimization](https://mlsanity.com/paper/2210.14808/) (2022)

## Built on by (most cited first, 3 of 3)

- [Improving OCR using internal document redundancy](https://mlsanity.com/paper/2508.14557/) (2025)
- [Deep Sprite-based Image Models: An Analysis](https://mlsanity.com/paper/2604.19480/) (2026)
- [Leveraging Morphology for Historical Script Metrological Analysis](https://mlsanity.com/paper/2606.09446/) (2026)

## Abstract (arXiv)

We present a generative document-specific approach to character analysis and recognition in text lines. Our main idea is to build on unsupervised multi-object segmentation methods and in particular those that reconstruct images based on a limited amount of visual elements, called sprites. Taking as input a set of text lines with similar font or handwriting, our approach can learn a large number of different characters and leverage line-level annotations when available. Our contribution is twofold. First, we provide the first adaptation and evaluation of a deep unsupervised multi-object segmentation approach for text line analysis. Since these methods have mainly been evaluated on synthetic data in a completely unsupervised setting, demonstrating that they can be adapted and quantitatively evaluated on real images of text and that they can be trained using weak supervision are significant progresses. Second, we show the potential of our method for new applications, more specifically in the field of paleography, which studies the history and variations of handwriting, and for cipher analysis. We demonstrate our approach on three very different datasets: a printed volume of the Google1000 dataset, the Copiale cipher and historical handwritten charters from the 12th and early 13th century.