# Predicting tongue motion in unlabeled ultrasound videos using convolutional LSTM neural network

arXiv: https://arxiv.org/abs/1902.06927 · HTML: https://mlsanity.com/paper/1902.06927/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Feb 2019.

## Built on by (most cited first, 4 of 4)

- [3D Convolutional Neural Networks for Ultrasound-Based Silent Speech Interfaces](https://mlsanity.com/paper/2104.11532/) (2021)
- [Adaptation of Tongue Ultrasound-Based Silent Speech Interfaces Using Spatial Transformer Networks](https://mlsanity.com/paper/2305.19130/) (2023)
- [Convolutional Neural Network-Based Age Estimation Using B-Mode Ultrasound Tongue Image](https://mlsanity.com/paper/2101.11245/) (2021)
- [Convolutional Long Short-Term Memory (convLSTM) for Spatio-Temporal Forecastings of Saturations and Pressure in the SACROC Field](https://mlsanity.com/paper/2212.00796/) (2022)

## Abstract (arXiv)

A challenge in speech production research is to predict future tongue movements based on a short period of past tongue movements. This study tackles speaker-dependent tongue motion prediction problem in unlabeled ultrasound videos with convolutional long short-term memory (ConvLSTM) networks. The model has been tested on two different ultrasound corpora. ConvLSTM outperforms 3-dimensional convolutional neural network (3DCNN) in predicting the 9textsuperscriptth frames based on 8 preceding frames, and also demonstrates good capacity to predict only the tongue contours in future frames. Further tests reveal that ConvLSTM can also learn to predict tongue movements in more distant frames beyond the immediately following frames. Our codes are available at: https://github.com/shuiliwanwu/ConvLstm-ultrasound-videos.