# Exploiting ultrasound tongue imaging for the automatic detection of speech articulation errors

arXiv: https://arxiv.org/abs/2103.00324 · HTML: https://mlsanity.com/paper/2103.00324/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 58 references that are in the map.
- First posted on arXiv 27 Feb 2021.

## Builds on (most cited first, 7 of 7)

- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [UltraSuite: A Repository of Ultrasound and Acoustic Data from Child Speech Therapy Sessions](https://mlsanity.com/paper/1907.00835/) (2019)
- [TaL: a synchronised multi-speaker corpus of ultrasound tongue imaging, audio, and lip videos](https://mlsanity.com/paper/2011.09804/) (2020)
- [Speaker-independent classification of phonetic segments from raw ultrasound in child speech](https://mlsanity.com/paper/1907.01413/) (2019)
- [Synchronising audio and ultrasound by learning cross-modal embeddings](https://mlsanity.com/paper/1907.00758/) (2019)
- [Ultrasound tongue imaging for diarization and alignment of child speech therapy sessions](https://mlsanity.com/paper/1907.00818/) (2019)
- [Quantification of Transducer Misalignment in Ultrasound Tongue Imaging](https://mlsanity.com/paper/2008.02470/) (2020)

## Built on by (most cited first, 2 of 2)

- [Automatic audiovisual synchronisation for ultrasound tongue imaging](https://mlsanity.com/paper/2105.15162/) (2021)
- [An Audio-textual Diffusion Model For Converting Speech Signals Into Ultrasound Tongue Imaging Data](https://mlsanity.com/paper/2403.05820/) (2024)

## Abstract (arXiv)

Speech sound disorders are a common communication impairment in childhood. Because speech disorders can negatively affect the lives and the development of children, clinical intervention is often recommended. To help with diagnosis and treatment, clinicians use instrumented methods such as spectrograms or ultrasound tongue imaging to analyse speech articulations. Analysis with these methods can be laborious for clinicians, therefore there is growing interest in its automation. In this paper, we investigate the contribution of ultrasound tongue imaging for the automatic detection of speech articulation errors. Our systems are trained on typically developing child speech and augmented with a database of adult speech using audio and ultrasound. Evaluation on typically developing speech indicates that pre-training on adult speech and jointly using ultrasound and audio gives the best results with an accuracy of 86.9%. To evaluate on disordered speech, we collect pronunciation scores from experienced speech and language therapists, focusing on cases of velar fronting and gliding of /r/. The scores show good inter-annotator agreement for velar fronting, but not for gliding errors. For automatic velar fronting error detection, the best results are obtained when jointly using ultrasound and audio. The best system correctly detects 86.6% of the errors identified by experienced clinicians. Out of all the segments identified as errors by the best system, 73.2% match errors identified by clinicians. Results on automatic gliding detection are harder to interpret due to poor inter-annotator agreement, but appear promising. Overall findings suggest that automatic detection of speech articulation errors has potential to be integrated into ultrasound intervention software for automatically quantifying progress during speech therapy.