DSS: Synthesizing long Digital Ink using Data augmentation, Style encoding and Split generation
Not on the map. Why?
Abstract (arXiv)
As text generative models can give increasingly long answers, we tackle the problem of synthesizing long text in digital ink. We show that the commonly used models for this task fail to generalize to long-form data and how this problem can be solved by augmenting the training data, changing the model architecture and the inference procedure. These methods use contrastive learning technique and are tailored specifically for the handwriting domain. They can be applied to any encoder-decoder model that works with digital ink. We demonstrate that our method reduces the character error rate on long-form English data by half compared to baseline RNN and by 16% compared to the previous approach that aims at addressing the same problem. We show that all three parts of the method improve recognizability of generated inks. In addition, we evaluate synthesized data in a human study and find that people perceive most of generated data as real.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 26 of its 43 references that are in the map. First posted on arXiv 29 Nov 2023.
Builds on
26 of 43 references.
References · 26
- 2017Attention Is All You Needcited by 60,361
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
- 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
- 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,850
- 2022LaMDA: Language Models for Dialog Applicationscited by 1,220
- 2016Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyondcited by 812
- 2015Scheduled Sampling for Sequence Prediction with Recurrent Neural Networkscited by 803
- 2013Generating Sequences With Recurrent Neural Networkscited by 554
- 2017Six Challenges for Neural Machine Translationcited by 335
- 2016Professor Forcing: A New Algorithm for Training Recurrent Networkscited by 216
- 2022Exploring Length Generalization in Large Language Modelscited by 206
14 more
- 2020Sketchformer: Transformer-based Representation for Sketched Structurecited by 78
- 2019Recognizing long-form speech using streaming end-to-end modelscited by 77
- 2020B'ezierSketch: A generative model for scalable vector sketchescited by 43
- 2018DeepWriting: Making Digital Ink Editable via Deep Generative Modelingcited by 39
- 2019Adversarial Generation of Handwritten Text Images Conditioned on Sequencescited by 37
- 2021Sequence Length is a Domain: Length-based Overfitting in Transformer Modelscited by 35
- 2021Instant One-Shot Word-Learning for Context-Specific Neural Sequence-to-Sequence Speech Recognitioncited by 30
- 2022StyleCLIPDraw: Coupling Content and Style in Text-to-Drawing Translationcited by 28
- 2020RNN-T Models Fail to Generalize to Out-of-Domain Audio: Causes and Solutionscited by 26
- 2022SLOGAN: Handwriting Style Synthesis for Arbitrary-Length and Out-of-Vocabulary Textcited by 26
- 2019Fast Multi-language LSTM-based Online Handwriting Recognitioncited by 22
- 2018HWNet v2: An Efficient Word Image Representation for Handwritten Documentscited by 13
- 2021Input Length Matters: Improving RNN-T and MWER Training for Long-form Telephony Speech Recognitioncited by 10
- 2021Sentence Concatenation Approach to Data Augmentation for Neural Machine Translationcited by 8
Built on by
Citing papers per year: 2024: 1; 2025: 0; 2026: 1.
2 papers.