# Modality Conversion of Handwritten Patterns by Cross Variational Autoencoders

arXiv: https://arxiv.org/abs/1906.06142 · HTML: https://mlsanity.com/paper/1906.06142/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Jun 2019.

## Built on by (most cited first, 8 of 8)

- [Complex Handwriting Trajectory Recovery: Evaluation Metrics and Algorithm](https://mlsanity.com/paper/2210.15879/) (2022)
- [Shared Latent Space of Font Shapes and Their Noisy Impressions](https://mlsanity.com/paper/2103.12347/) (2021)
- [TRACE: A Differentiable Approach to Line-level Stroke Recovery for Offline Handwritten Text](https://mlsanity.com/paper/2105.11559/) (2021)
- [Writing Order Recovery in Complex and Long Static Handwriting](https://mlsanity.com/paper/2406.03194/) (2024)
- [InkSight: Offline-to-Online Handwriting Conversion by Teaching Vision-Language Models to Read and Write](https://mlsanity.com/paper/2402.05804/) (2024)
- [Self-Augmented Multi-Modal Feature Embedding](https://mlsanity.com/paper/2103.04731/) (2021)
- [Handwriting Trajectory Recovery with Diffusion Models](https://mlsanity.com/paper/2607.03422/) (2026)
- [Handwriting Trajectory Recovery via Autoregressive Ordered Stroke Instance Prediction](https://mlsanity.com/paper/2609.02251/) (2026)

## Abstract (arXiv)

This research attempts to construct a network that can convert online and offline handwritten characters to each other. The proposed network consists of two Variational Auto-Encoders (VAEs) with a shared latent space. The VAEs are trained to generate online and offline handwritten Latin characters simultaneously. In this way, we create a cross-modal VAE (Cross-VAE). During training, the proposed Cross-VAE is trained to minimize the reconstruction loss of the two modalities, the distribution loss of the two VAEs, and a novel third loss called the space sharing loss. This third, space sharing loss is used to encourage the modalities to share the same latent space by calculating the distance between the latent variables. Through the proposed method mutual conversion of online and offline handwritten characters is possible. In this paper, we demonstrate the performance of the Cross-VAE through qualitative and quantitative analysis.