# Scalable Font Reconstruction with Dual Latent Manifolds

arXiv: https://arxiv.org/abs/2109.06627 · HTML: https://mlsanity.com/paper/2109.06627/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 39 references that are in the map.
- First posted on arXiv 10 Sep 2021.

## Builds on (most cited first, 24 of 24)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016)
- [NICE: Non-linear Independent Components Estimation](https://mlsanity.com/paper/1410.8516/) (2014)
- [HyperNetworks](https://mlsanity.com/paper/1609.09106/) (2016)
- [A Neural Algorithm of Artistic Style](https://mlsanity.com/paper/1508.06576/) (2015)
- [Making Convolutional Networks Shift-Invariant Again](https://mlsanity.com/paper/1904.11486/) (2019)
- [A General and Adaptive Robust Loss Function](https://mlsanity.com/paper/1701.03077/) (2017)
- [Texture Networks: Feed-forward Synthesis of Textures and Stylized Images](https://mlsanity.com/paper/1603.03417/) (2016)
- [Multi-Content GAN for Few-Shot Font Style Transfer](https://mlsanity.com/paper/1712.00516/) (2017)
- [A Learned Representation for Scalable Vector Graphics](https://mlsanity.com/paper/1904.02632/) (2019)
- [StyleBank: An Explicit Representation for Neural Image Style Transfer](https://mlsanity.com/paper/1703.09210/) (2017)
- [Separating Style and Content for Generalized Style Transfer](https://mlsanity.com/paper/1711.06454/) (2017)
- [Artistic Glyph Image Synthesis via One-Stage Few-Shot Learning](https://mlsanity.com/paper/1910.04987/) (2019)
- [Generative Image Modeling using Style and Structure Adversarial Networks](https://mlsanity.com/paper/1603.05631/) (2016)
- [Learning to Generate Images with Perceptual Similarity Metrics](https://mlsanity.com/paper/1511.06409/) (2015)
- [Style and Content Disentanglement in Generative Adversarial Networks](https://mlsanity.com/paper/1811.05621/) (2018)
- [Retrieval-Enhanced Adversarial Training for Neural Response Generation](https://mlsanity.com/paper/1809.04276/) (2018)
- [A Deep Factorization of Style and Structure in Fonts](https://mlsanity.com/paper/1910.00748/) (2019)
- [A Unified Framework for Generalizable Style Transfer: Style and Content Separation](https://mlsanity.com/paper/1806.05173/) (2018)

## Built on by (most cited first, 4 of 4)

- [Font Style Interpolation with Diffusion Models](https://mlsanity.com/paper/2402.14311/) (2024)
- [Contour Completion by Transformers and Its Application to Vector Font Data](https://mlsanity.com/paper/2304.13988/) (2023)
- [TrueType Transformer: Character and Font Style Recognition in Outline Format](https://mlsanity.com/paper/2203.05338/) (2022)
- [Total Disentanglement of Font Images into Style and Character Class Features](https://mlsanity.com/paper/2403.12784/) (2024)

## Abstract (arXiv)

We propose a deep generative model that performs typography analysis and font reconstruction by learning disentangled manifolds of both font style and character shape. Our approach enables us to massively scale up the number of character types we can effectively model compared to previous methods. Specifically, we infer separate latent variables representing character and font via a pair of inference networks which take as input sets of glyphs that either all share a character type, or belong to the same font. This design allows our model to generalize to characters that were not observed during training time, an important task in light of the relative sparsity of most fonts. We also put forward a new loss, adapted from prior work that measures likelihood using an adaptive distribution in a projected space, resulting in more natural images without requiring a discriminator. We evaluate on the task of font reconstruction over various datasets representing character types of many languages, and compare favorably to modern style transfer systems according to both automatic and manually-evaluated metrics.