# An Effective Automated Speaking Assessment Approach to Mitigating Data Scarcity and Imbalanced Distribution

arXiv: https://arxiv.org/abs/2404.07575 · HTML: https://mlsanity.com/paper/2404.07575/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 36 references that are in the map.
- First posted on arXiv 11 Apr 2024.

## Builds on (most cited first, 13 of 13)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Cross-lingual Language Model Pretraining](https://mlsanity.com/paper/1901.07291/) (2019)
- [In defence of metric learning for speaker recognition](https://mlsanity.com/paper/2003.11982/) (2020)
- [Multi-Level Matching and Aggregation Network for Few-Shot Relation Classification](https://mlsanity.com/paper/1906.06678/) (2019)
- [Automatic Pronunciation Assessment using Self-Supervised Speech Representation Learning](https://mlsanity.com/paper/2204.03863/) (2022)
- [CEFR-Based Sentence Difficulty Annotation and Assessment](https://mlsanity.com/paper/2210.11766/) (2022)
- [3M: An Effective Multi-view, Multi-granularity, and Multi-aspect Modeling Approach to English Pronunciation Assessment](https://mlsanity.com/paper/2208.09110/) (2022)
- [Proficiency assessment of L2 spoken English using wav2vec 2.0](https://mlsanity.com/paper/2210.13168/) (2022)
- [A Hierarchical Context-aware Modeling Approach for Multi-aspect and Multi-granular Pronunciation Assessment](https://mlsanity.com/paper/2305.18146/) (2023)

## Built on by (most cited first, 7 of 7)

- [The NTNU System at the S&I Challenge 2025 SLA Open Track](https://mlsanity.com/paper/2506.05121/) (2025)
- [Zero-Shot Text-to-Speech as Golden Speech Generator: A Systematic Framework and its Applicability in Automatic Pronunciation Assessment](https://mlsanity.com/paper/2409.07151/) (2024)
- [Automatic Proficiency Assessment in L2 English Learners](https://mlsanity.com/paper/2505.02615/) (2025)
- [A Novel Data Augmentation Approach for Automatic Speaking Assessment on Opinion Expressions](https://mlsanity.com/paper/2506.04077/) (2025)
- [Beyond Modality Limitations: A Unified MLLM Approach to Automated Speaking Assessment with Effective Curriculum Learning](https://mlsanity.com/paper/2508.12591/) (2025)
- [Mitigating Data Imbalance in Automated Speaking Assessment](https://mlsanity.com/paper/2509.03010/) (2025)
- [An Effective Strategy for Modeling Score Ordinality and Non-uniform Intervals in Automated Speaking Assessment](https://mlsanity.com/paper/2509.03372/) (2025)

## Abstract (arXiv)

Automated speaking assessment (ASA) typically involves automatic speech recognition (ASR) and hand-crafted feature extraction from the ASR transcript of a learner's speech. Recently, self-supervised learning (SSL) has shown stellar performance compared to traditional methods. However, SSL-based ASA systems are faced with at least three data-related challenges: limited annotated data, uneven distribution of learner proficiency levels and non-uniform score intervals between different CEFR proficiency levels. To address these challenges, we explore the use of two novel modeling strategies: metric-based classification and loss reweighting, leveraging distinct SSL-based embedding features. Extensive experimental results on the ICNALE benchmark dataset suggest that our approach can outperform existing strong baselines by a sizable margin, achieving a significant improvement of more than 10% in CEFR prediction accuracy.