# Phonetic Error Analysis of Raw Waveform Acoustic Models with Parametric and Non-Parametric CNNs

arXiv: https://arxiv.org/abs/2406.00898 · HTML: https://mlsanity.com/paper/2406.00898/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 44 references that are in the map.
- First posted on arXiv 2 Jun 2024.

## Builds on (most cited first, 11 of 11)

- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Phase-aware Speech Enhancement with Deep Complex U-Net](https://mlsanity.com/paper/1903.03107/) (2019)
- [The PyTorch-Kaldi Speech Recognition Toolkit](https://mlsanity.com/paper/1811.07453/) (2018)
- [Learning Multiscale Features Directly From Waveforms](https://mlsanity.com/paper/1603.09509/) (2016)
- [CGCNN: Complex Gabor Convolutional Neural Network on raw speech](https://mlsanity.com/paper/2002.04569/) (2020)
- [The Role of Phonetic Units in Speech Emotion Recognition](https://mlsanity.com/paper/2108.01132/) (2021)
- [Incorporating Broad Phonetic Information for Speech Enhancement](https://mlsanity.com/paper/2008.07618/) (2020)
- [Learning Waveform-Based Acoustic Models using Deep Variational Convolutional Neural Networks](https://mlsanity.com/paper/1906.09526/) (2019)
- [Multi-Span Acoustic Modelling using Raw Waveform Signals](https://mlsanity.com/paper/1906.11047/) (2019)
- [Acoustic Model Adaptation from Raw Waveforms with SincNet](https://mlsanity.com/paper/1909.13759/) (2019)

## Abstract (arXiv)

In this paper, we analyse the error patterns of the raw waveform acoustic models in TIMIT's phone recognition task. Our analysis goes beyond the conventional phone error rate (PER) metric. We categorise the phones into three groups: {affricate, diphthong, fricative, nasal, plosive, semi-vowel, vowel, silence}, {consonant, vowel+, silence}, and {voiced, unvoiced, silence} and, compute the PER for each broad phonetic class in each category. We also construct a confusion matrix for each category using the substitution errors and compare the confusion patterns with those of the Filterbank and Wav2vec 2.0 systems. Our raw waveform acoustic models consists of parametric (Sinc2Net) or non-parametric CNNs and Bidirectional LSTMs, achieving down to 13.7%/15.2% PERs on TIMIT Dev/Test sets, outperforming reported PERs for raw waveform models in the literature. We also investigate the impact of transfer learning from WSJ on the phonetic error patterns and confusion matrices. It reduces the PER to 11.8%/13.7% on the Dev/Test sets.