# TaL: a synchronised multi-speaker corpus of ultrasound tongue imaging, audio, and lip videos

arXiv: https://arxiv.org/abs/2011.09804 · HTML: https://mlsanity.com/paper/2011.09804/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Nov 2020.

## Built on by (most cited first, 12 of 12)

- [Exploring Self-supervised Pre-trained ASR Models For Dysarthric and Elderly Speech Recognition](https://mlsanity.com/paper/2302.14564/) (2023)
- [Self-supervised ASR Models and Features For Dysarthric and Elderly Speech Recognition](https://mlsanity.com/paper/2407.13782/) (2024)
- [Exploiting Cross-domain And Cross-Lingual Ultrasound Tongue Imaging Features For Elderly And Dysarthric Speech Recognition](https://mlsanity.com/paper/2206.07327/) (2022)
- [Exploiting ultrasound tongue imaging for the automatic detection of speech articulation errors](https://mlsanity.com/paper/2103.00324/) (2021)
- [Adaptation of Tacotron2-based Text-To-Speech for Articulatory-to-Acoustic Mapping using Ultrasound Tongue Imaging](https://mlsanity.com/paper/2107.12051/) (2021)
- [UltraUNet: Real-Time Ultrasound Tongue Segmentation for Diverse Linguistic and Imaging Conditions](https://mlsanity.com/paper/2509.23225/) (2025)
- [Adaptation of Tongue Ultrasound-Based Silent Speech Interfaces Using Spatial Transformer Networks](https://mlsanity.com/paper/2305.19130/) (2023)
- [Silent versus modal multi-speaker speech recognition from ultrasound and video](https://mlsanity.com/paper/2103.00333/) (2021)
- [Automatic audiovisual synchronisation for ultrasound tongue imaging](https://mlsanity.com/paper/2105.15162/) (2021)
- [An Audio-textual Diffusion Model For Converting Speech Signals Into Ultrasound Tongue Imaging Data](https://mlsanity.com/paper/2403.05820/) (2024)
- [SoniSpeech: A Large-Scale Open-Vocabulary Tri-Modal Dataset for Wearable Silent Speech Interfaces](https://mlsanity.com/paper/2608.00803/) (2026)
- [Dual Co-Train: Cross-Dataset Ultrasound Tongue Segmentation Under Extreme Data Scarcity](https://mlsanity.com/paper/2608.17983/) (2026)

## Abstract (arXiv)

We present the Tongue and Lips corpus (TaL), a multi-speaker corpus of audio, ultrasound tongue imaging, and lip videos. TaL consists of two parts: TaL1 is a set of six recording sessions of one professional voice talent, a male native speaker of English; TaL80 is a set of recording sessions of 81 native speakers of English without voice talent experience. Overall, the corpus contains 24 hours of parallel ultrasound, video, and audio data, of which approximately 13.5 hours are speech. This paper describes the corpus and presents benchmark results for the tasks of speech recognition, speech synthesis (articulatory-to-acoustic mapping), and automatic synchronisation of ultrasound to audio. The TaL corpus is publicly available under the CC BY-NC 4.0 license.