# Rethinking HTG Evaluation: Bridging Generation and Recognition

arXiv: https://arxiv.org/abs/2409.02683 · HTML: https://mlsanity.com/paper/2409.02683/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 34 references that are in the map.
- First posted on arXiv 4 Sep 2024.

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [How good is my GAN?](https://mlsanity.com/paper/1807.09499/) (2018)
- [ScrabbleGAN: Semi-Supervised Varying Length Handwritten Text Generation](https://mlsanity.com/paper/2003.10557/) (2020)
- [GANwriting: Content-Conditioned Generation of Styled Handwritten Word Images](https://mlsanity.com/paper/2003.02567/) (2020)
- [What If We Only Use Real Datasets for Scene Text Recognition? Toward Scene Text Recognition With Fewer Labels](https://mlsanity.com/paper/2103.04400/) (2021)
- [Conditional Text Image Generation with Diffusion Models](https://mlsanity.com/paper/2306.10804/) (2023)
- [Geometry Score: A Method For Comparing Generative Adversarial Networks](https://mlsanity.com/paper/1802.02664/) (2018)
- [TextStyleBrush: Transfer of Text Aesthetics from a Single Example](https://mlsanity.com/paper/2106.08385/) (2021)
- [Adversarial Generation of Handwritten Text Images Conditioned on Sequences](https://mlsanity.com/paper/1903.00277/) (2019)
- [Text and Style Conditioned GAN for Generation of Offline Handwriting Lines](https://mlsanity.com/paper/2009.00678/) (2020)
- [Content and Style Aware Generation of Text-line Images for Handwriting Recognition](https://mlsanity.com/paper/2204.05539/) (2022)
- [Handwritten Text Generation from Visual Archetypes](https://mlsanity.com/paper/2303.15269/) (2023)
- [SLOGAN: Handwriting Style Synthesis for Arbitrary-Length and Out-of-Vocabulary Text](https://mlsanity.com/paper/2202.11456/) (2022)
- [WordStylist: Styled Verbatim Handwritten Text Generation with Latent Diffusion Models](https://mlsanity.com/paper/2303.16576/) (2023)
- [A Survey of Historical Document Image Datasets](https://mlsanity.com/paper/2203.08504/) (2022)
- [HWD: A Novel Evaluation Score for Styled Handwritten Text Generation](https://mlsanity.com/paper/2310.20316/) (2023)
- [VATr++: Choose Your Words Wisely for Handwritten Text Generation](https://mlsanity.com/paper/2402.10798/) (2024)
- [SmartPatch: Improving Handwritten Word Imitation with Patch Discriminators](https://mlsanity.com/paper/2105.10528/) (2021)
- [Best Practices for a Handwritten Text Recognition System](https://mlsanity.com/paper/2404.11339/) (2024)
- [Temporal Pyramid Pooling Based Convolutional Neural Networks for Action Recognition](https://mlsanity.com/paper/1503.01224/) (2015)
- [A Systematic Performance Analysis of Deep Perceptual Loss Networks: Breaking Transfer Learning Conventions](https://mlsanity.com/paper/2302.04032/) (2023)

## Built on by (most cited first, 6 of 6)

- [Zero-Shot Styled Text Image Generation, but Make It Autoregressive](https://mlsanity.com/paper/2503.17074/) (2025)
- [Beyond Isolated Words: Diffusion Brush for Handwritten Text-Line Generation](https://mlsanity.com/paper/2508.03256/) (2025)
- [Beyond Human Annotation: Recent Advances in Data Generation Methods for Document Intelligence](https://mlsanity.com/paper/2601.12318/) (2026)
- [Quo Vadis Handwritten Text Generation for Handwritten Text Recognition?](https://mlsanity.com/paper/2508.09936/) (2025)
- [Dual Orthogonal Guidance for Robust Diffusion-based Handwritten Text Generation](https://mlsanity.com/paper/2508.17017/) (2025)
- [CONSTANT: Towards High-Quality One-Shot Handwriting Generation with Patch Contrastive Enhancement and Style-Aware Quantization](https://mlsanity.com/paper/2603.07543/) (2026)

## Abstract (arXiv)

The evaluation of generative models for natural image tasks has been extensively studied. Similar protocols and metrics are used in cases with unique particularities, such as Handwriting Generation, even if they might not be completely appropriate. In this work, we introduce three measures tailored for HTG evaluation,  HTGHTR ,  HTGstyle , and  HTGOOV , and argue that they are more expedient to evaluate the quality of generated handwritten images. The metrics rely on the recognition error/accuracy of Handwriting Text Recognition and Writer Identification models and emphasize writing style, textual content, and diversity as the main aspects that adhere to the content of handwritten images. We conduct comprehensive experiments on the IAM handwriting database, showcasing that widely used metrics such as FID fail to properly quantify the diversity and the practical utility of generated handwriting samples. Our findings show that our metrics are richer in information and underscore the necessity of standardized evaluation protocols in HTG. The proposed metrics provide a more robust and informative protocol for assessing HTG quality, contributing to improved performance in HTR. Code for the evaluation protocol is available at: https://github.com/koninik/HTGevaluation.