# Replacing Human Audio with Synthetic Audio for On-device Unspoken Punctuation Prediction

arXiv: https://arxiv.org/abs/2010.10203 · HTML: https://mlsanity.com/paper/2010.10203/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Oct 2020.

## Built on by (most cited first, 8 of 8)

- [Capitalization and Punctuation Restoration: a Survey](https://mlsanity.com/paper/2111.10746/) (2021)
- [Cross-Modal ASR Post-Processing System for Error Correction and Utterance Rejection](https://mlsanity.com/paper/2201.03313/) (2022)
- [Unified Multimodal Punctuation Restoration Framework for Mixed-Modality Corpus](https://mlsanity.com/paper/2202.00468/) (2022)
- [Correcting Mispronunciations in Speech using Spectrogram Inpainting](https://mlsanity.com/paper/2204.03379/) (2022)
- [Improved Long-Form Spoken Language Translation with Large Language Models](https://mlsanity.com/paper/2212.09895/) (2022)
- [Exploring the Viability of Synthetic Audio Data for Audio-Based Dialogue State Tracking](https://mlsanity.com/paper/2312.01842/) (2023)
- [Contrastive Learning from Synthetic Audio Doppelg"angers](https://mlsanity.com/paper/2406.05923/) (2024)
- [Overcoming Latency Bottlenecks in On-Device Speech Translation: A Cascaded Approach with Alignment-Based Streaming MT](https://mlsanity.com/paper/2508.13358/) (2025)

## Abstract (arXiv)

We present a novel multi-modal unspoken punctuation prediction system for the English language which combines acoustic and text features. We demonstrate for the first time, that by relying exclusively on synthetic data generated using a prosody-aware text-to-speech system, we can outperform a model trained with expensive human audio recordings on the unspoken punctuation prediction problem. Our model architecture is well suited for on-device use. This is achieved by leveraging hash-based embeddings of automatic speech recognition text output in conjunction with acoustic features as input to a quasi-recurrent neural network, keeping the model size small and latency low.