# Interpretable Modeling of Articulatory Temporal Dynamics from real-time MRI for Phoneme Recognition

arXiv: https://arxiv.org/abs/2509.15689 · HTML: https://mlsanity.com/paper/2509.15689/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Sep 2025.

## Built on by (most cited first, 3 of 3)

- [An Approach to Simultaneous Acquisition of Real-Time MRI Video, EEG, and Surface EMG for Articulatory, Brain, and Muscle Activity During Speech Production](https://mlsanity.com/paper/2603.04840/) (2026)
- [Structured Phonological Representations for Audio-Articulatory rtMRI Speech Classification](https://mlsanity.com/paper/2608.09767/) (2026)
- [Arti-JEPA: Adapting Video World Model to Real-Time MRI of the Vocal Tract for Speech-Production Analysis](https://mlsanity.com/paper/2609.09757/) (2026)

## Abstract (arXiv)

Real-time Magnetic Resonance Imaging (rtMRI) visualizes vocal tract action, offering a comprehensive window into speech articulation. However, its signals are high dimensional and noisy, hindering interpretation. We investigate compact representations of spatiotemporal articulatory dynamics for phoneme recognition from midsagittal vocal tract rtMRI videos. We compare three feature types: (1) raw video, (2) optical flow, and (3) six linguistically-relevant regions of interest (ROIs) for articulator movements. We evaluate models trained independently on each representation, as well as multi-feature combinations. Results show that multi-feature models consistently outperform single-feature baselines, with the lowest phoneme error rate (PER) of 0.34 obtained by combining ROI and raw video. Temporal fidelity experiments demonstrate a reliance on fine-grained articulatory dynamics, while ROI ablation studies reveal strong contributions from tongue and lips. Our findings highlight how rtMRI-derived features provide accuracy and interpretability, and establish strategies for leveraging articulatory data in speech processing.