# JVNV: A Corpus of Japanese Emotional Speech with Verbal Content and Nonverbal Expressions

arXiv: https://arxiv.org/abs/2310.06072 · HTML: https://mlsanity.com/paper/2310.06072/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Oct 2023.

## Built on by (most cited first, 10 of 10)

- [Laugh Now Cry Later: Controlling Time-Varying Emotional States of Flow-Matching-Based Zero-Shot Text-to-Speech](https://mlsanity.com/paper/2407.12229/) (2024)
- [NonverbalTTS: A Public English Corpus of Text-Aligned Nonverbal Vocalizations with Emotion Annotations for Text-to-Speech](https://mlsanity.com/paper/2507.13155/) (2025)
- [MNV-17: A High-Quality Performative Mandarin Dataset for Nonverbal Vocalization Recognition in Speech](https://mlsanity.com/paper/2509.18196/) (2025)
- [Remastering Divide and Remaster: A Cinematic Audio Source Separation Dataset with Multilingual Support](https://mlsanity.com/paper/2407.07275/) (2024)
- [J-CHAT: Japanese Large-scale Spoken Dialogue Corpus for Spoken Dialogue Language Modeling](https://mlsanity.com/paper/2407.15828/) (2024)
- [AUDDT: A Unified Benchmark Toolkit for Audio and Speech Deepfake Detectors](https://mlsanity.com/paper/2509.21597/) (2025)
- [Performance Evaluation of Emotion Classification in Japanese Using RoBERTa and DeBERTa](https://mlsanity.com/paper/2505.00013/) (2025)
- [Are Mamba-based Audio Foundation Models the Best Fit for Non-Verbal Emotion Recognition?](https://mlsanity.com/paper/2506.02258/) (2025)
- [Color-based Emotion Representation for Speech Emotion Recognition](https://mlsanity.com/paper/2602.16256/) (2026)
- [MoVE: Translating Laughter and Tears via Mixture of Vocalization Experts in Speech-to-Speech Translation](https://mlsanity.com/paper/2604.17435/) (2026)

## Abstract (arXiv)

We present the JVNV, a Japanese emotional speech corpus with verbal content and nonverbal vocalizations whose scripts are generated by a large-scale language model. Existing emotional speech corpora lack not only proper emotional scripts but also nonverbal vocalizations (NVs) that are essential expressions in spoken language to express emotions. We propose an automatic script generation method to produce emotional scripts by providing seed words with sentiment polarity and phrases of nonverbal vocalizations to ChatGPT using prompt engineering. We select 514 scripts with balanced phoneme coverage from the generated candidate scripts with the assistance of emotion confidence scores and language fluency scores. We demonstrate the effectiveness of JVNV by showing that JVNV has better phoneme coverage and emotion recognizability than previous Japanese emotional speech corpora. We then benchmark JVNV on emotional text-to-speech synthesis using discrete codes to represent NVs. We show that there still exists a gap between the performance of synthesizing read-aloud speech and emotional speech, and adding NVs in the speech makes the task even harder, which brings new challenges for this task and makes JVNV a valuable resource for relevant works in the future. To our best knowledge, JVNV is the first speech corpus that generates scripts automatically using large language models.