# A Novel Data Augmentation Approach for Automatic Speaking Assessment on Opinion Expressions

arXiv: https://arxiv.org/abs/2506.04077 · HTML: https://mlsanity.com/paper/2506.04077/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 28 references that are in the map.
- First posted on arXiv 4 Jun 2025.

## Builds on (most cited first, 14 of 14)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs](https://mlsanity.com/paper/2503.01743/) (2025)
- [GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation](https://mlsanity.com/paper/2104.08826/) (2021)
- [Data Augmentation for Intent Classification with Off-the-shelf Large Language Models](https://mlsanity.com/paper/2204.01959/) (2022)
- [SynthASR: Unlocking Synthetic Data for Speech Recognition](https://mlsanity.com/paper/2106.07803/) (2021)
- [EmoMix: Emotion Mixing via Diffusion Models for Emotional Speech Synthesis](https://mlsanity.com/paper/2306.00648/) (2023)
- [Text is All You Need: Personalizing ASR Models using Controllable Speech Synthesis](https://mlsanity.com/paper/2303.14885/) (2023)
- [An Effective Automated Speaking Assessment Approach to Mitigating Data Scarcity and Imbalanced Distribution](https://mlsanity.com/paper/2404.07575/) (2024)
- [Not All LLM-Generated Data Are Equal: Rethinking Data Weighting in Text Classification](https://mlsanity.com/paper/2410.21526/) (2024)
- [A knowledge-driven vowel-based approach of depression classification from speech using data augmentation](https://mlsanity.com/paper/2210.15261/) (2022)
- [Addressing Cold Start Problem for End-to-end Automatic Speech Scoring](https://mlsanity.com/paper/2306.14310/) (2023)

## Abstract (arXiv)

Automated speaking assessment (ASA) on opinion expressions is often hampered by the scarcity of labeled recordings, which restricts prompt diversity and undermines scoring reliability. To address this challenge, we propose a novel training paradigm that leverages a large language models (LLM) to generate diverse responses of a given proficiency level, converts responses into synthesized speech via speaker-aware text-to-speech synthesis, and employs a dynamic importance loss to adaptively reweight training instances based on feature distribution differences between synthesized and real speech. Subsequently, a multimodal large language model integrates aligned textual features with speech signals to predict proficiency scores directly. Experiments conducted on the LTTC dataset show that our approach outperforms methods relying on real data or conventional augmentation, effectively mitigating low-resource constraints and enabling ASA on opinion expressions with cross-modal information.