# DSS: Synthesizing long Digital Ink using Data augmentation, Style encoding and Split generation

arXiv: https://arxiv.org/abs/2311.17786 · HTML: https://mlsanity.com/paper/2311.17786/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 43 references that are in the map.
- First posted on arXiv 29 Nov 2023.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [LaMDA: Language Models for Dialog Applications](https://mlsanity.com/paper/2201.08239/) (2022)
- [Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond](https://mlsanity.com/paper/1602.06023/) (2016)
- [Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks](https://mlsanity.com/paper/1506.03099/) (2015)
- [Generating Sequences With Recurrent Neural Networks](https://mlsanity.com/paper/1308.0850/) (2013)
- [Six Challenges for Neural Machine Translation](https://mlsanity.com/paper/1706.03872/) (2017)
- [Professor Forcing: A New Algorithm for Training Recurrent Networks](https://mlsanity.com/paper/1610.09038/) (2016)
- [Exploring Length Generalization in Large Language Models](https://mlsanity.com/paper/2207.04901/) (2022)
- [Sketchformer: Transformer-based Representation for Sketched Structure](https://mlsanity.com/paper/2002.10381/) (2020)
- [Recognizing long-form speech using streaming end-to-end models](https://mlsanity.com/paper/1910.11455/) (2019)
- [B'ezierSketch: A generative model for scalable vector sketches](https://mlsanity.com/paper/2007.02190/) (2020)
- [DeepWriting: Making Digital Ink Editable via Deep Generative Modeling](https://mlsanity.com/paper/1801.08379/) (2018)
- [Adversarial Generation of Handwritten Text Images Conditioned on Sequences](https://mlsanity.com/paper/1903.00277/) (2019)
- [Sequence Length is a Domain: Length-based Overfitting in Transformer Models](https://mlsanity.com/paper/2109.07276/) (2021)
- [Instant One-Shot Word-Learning for Context-Specific Neural Sequence-to-Sequence Speech Recognition](https://mlsanity.com/paper/2107.02268/) (2021)
- [StyleCLIPDraw: Coupling Content and Style in Text-to-Drawing Translation](https://mlsanity.com/paper/2202.12362/) (2022)
- [RNN-T Models Fail to Generalize to Out-of-Domain Audio: Causes and Solutions](https://mlsanity.com/paper/2005.03271/) (2020)
- [SLOGAN: Handwriting Style Synthesis for Arbitrary-Length and Out-of-Vocabulary Text](https://mlsanity.com/paper/2202.11456/) (2022)
- [Fast Multi-language LSTM-based Online Handwriting Recognition](https://mlsanity.com/paper/1902.10525/) (2019)
- [HWNet v2: An Efficient Word Image Representation for Handwritten Documents](https://mlsanity.com/paper/1802.06194/) (2018)
- [Input Length Matters: Improving RNN-T and MWER Training for Long-form Telephony Speech Recognition](https://mlsanity.com/paper/2110.03841/) (2021)
- [Sentence Concatenation Approach to Data Augmentation for Neural Machine Translation](https://mlsanity.com/paper/2104.08478/) (2021)

## Built on by (most cited first, 2 of 2)

- [MathWriting: A Dataset For Handwritten Mathematical Expression Recognition](https://mlsanity.com/paper/2404.10690/) (2024)
- [Beyond Human Annotation: Recent Advances in Data Generation Methods for Document Intelligence](https://mlsanity.com/paper/2601.12318/) (2026)

## Abstract (arXiv)

As text generative models can give increasingly long answers, we tackle the problem of synthesizing long text in digital ink. We show that the commonly used models for this task fail to generalize to long-form data and how this problem can be solved by augmenting the training data, changing the model architecture and the inference procedure. These methods use contrastive learning technique and are tailored specifically for the handwriting domain. They can be applied to any encoder-decoder model that works with digital ink. We demonstrate that our method reduces the character error rate on long-form English data by half compared to baseline RNN and by 16% compared to the previous approach that aims at addressing the same problem. We show that all three parts of the method improve recognizability of generated inks. In addition, we evaluate synthesized data in a human study and find that people perceive most of generated data as real.