# Low-Resource Spoken Language Identification Using Self-Attentive Pooling and Deep 1D Time-Channel Separable Convolutions

arXiv: https://arxiv.org/abs/2106.00052 · HTML: https://mlsanity.com/paper/2106.00052/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 36 references that are in the map.
- First posted on arXiv 31 May 2021.

## Builds on (most cited first, 15 of 15)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty](https://mlsanity.com/paper/1912.02781/) (2019)
- [VoxLingua107: a Dataset for Spoken Language Recognition](https://mlsanity.com/paper/2011.12998/) (2020)
- [QuartzNet: Deep Automatic Speech Recognition with 1D Time-Channel Separable Convolutions](https://mlsanity.com/paper/1910.10261/) (2019)
- [Exploring the Encoding Layer and Loss Function in End-to-End Speaker and Language Recognition System](https://mlsanity.com/paper/1804.05160/) (2018)
- [Deep Representation Learning in Speech Processing: Challenges, Recent Advances, and Future Trends](https://mlsanity.com/paper/2001.00378/) (2020)
- [Learning Efficient Representations for Keyword Spotting with Triplet Loss](https://mlsanity.com/paper/2101.04792/) (2021)
- [Attention-Based Models for Text-Dependent Speaker Verification](https://mlsanity.com/paper/1710.10470/) (2017)
- [SpeakerNet: 1D Depth-wise Separable Convolutional Network for Text-Independent Speaker Recognition and Verification](https://mlsanity.com/paper/2010.12653/) (2020)
- [Language Identification Using Deep Convolutional Recurrent Neural Networks](https://mlsanity.com/paper/1708.04811/) (2017)
- [Spoken Language Identification using ConvNets](https://mlsanity.com/paper/1910.04269/) (2019)
- [Triplet Entropy Loss: Improving The Generalisation of Short Speech Language Identification Systems](https://mlsanity.com/paper/2012.03775/) (2020)
- [Cross-Domain Adaptation of Spoken Language Identification for Related Languages: The Curious Case of Slavic Languages](https://mlsanity.com/paper/2008.00545/) (2020)

## Abstract (arXiv)

This memo describes NTR/TSU winning submission for Low Resource ASR challenge at Dialog2021 conference, language identification track. Spoken Language Identification (LID) is an important step in a multilingual Automated Speech Recognition (ASR) system pipeline. Traditionally, the ASR task requires large volumes of labeled data that are unattainable for most of the world's languages, including most of the languages of Russia. In this memo, we show that a convolutional neural network with a Self-Attentive Pooling layer shows promising results in low-resource setting for the language identification task and set up a SOTA for the Low Resource ASR challenge dataset. Additionally, we compare the structure of confusion matrices for this and significantly more diverse VoxForge dataset and state and substantiate the hypothesis that whenever the dataset is diverse enough so that the other classification factors, like gender, age etc. are well-averaged, the confusion matrix for LID system bears the language similarity measure.