# Adaptive Activation Network For Low Resource Multilingual Speech Recognition

arXiv: https://arxiv.org/abs/2205.14326 · HTML: https://mlsanity.com/paper/2205.14326/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 37 references that are in the map.
- First posted on arXiv 28 May 2022.

## Builds on (most cited first, 18 of 18)

- [Deep Speech 2: End-to-End Speech Recognition in English and Mandarin](https://mlsanity.com/paper/1512.02595/) (2015)
- [Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learning](https://mlsanity.com/paper/1609.06773/) (2016)
- [Large-Scale Multilingual Speech Recognition with a Streaming End-to-End Model](https://mlsanity.com/paper/1909.05330/) (2019)
- [Meta Learning for End-to-End Low-Resource Speech Recognition](https://mlsanity.com/paper/1910.12094/) (2019)
- [Transfer Learning for Speech Recognition on a Budget](https://mlsanity.com/paper/1706.00290/) (2017)
- [Meta-Transfer Learning for Code-Switched Speech Recognition](https://mlsanity.com/paper/2004.14228/) (2020)
- [Sequence-based Multi-lingual Low Resource Speech Recognition](https://mlsanity.com/paper/1802.07420/) (2018)
- [DARTS-ASR: Differentiable Architecture Search for Multilingual Speech Recognition and Adaptation](https://mlsanity.com/paper/2005.07029/) (2020)
- [Adversarial Meta Sampling for Multilingual Low-Resource Speech Recognition](https://mlsanity.com/paper/2012.11896/) (2020)
- [Multilingual Graphemic Hybrid ASR with Massive Data Augmentation](https://mlsanity.com/paper/1909.06522/) (2019)
- [Very Deep Multilingual Convolutional Neural Networks for LVCSR](https://mlsanity.com/paper/1509.08967/) (2015)
- [Large-scale Transfer Learning for Low-resource Spoken Language Understanding](https://mlsanity.com/paper/2008.05671/) (2020)
- [Adaptive Activation Network and Functional Regularization for Efficient and Flexible Deep Multi-Task Learning](https://mlsanity.com/paper/1911.08065/) (2019)
- [Dropout Regularization for Self-Supervised Learning of Transformer Encoder Speech Representation](https://mlsanity.com/paper/2107.04227/) (2021)
- [Multilingual Speech Recognition with Corpus Relatedness Sampling](https://mlsanity.com/paper/1908.01060/) (2019)
- [Multi-QuartzNet: Multi-Resolution Convolution for Speech Recognition with Multi-Layer Feature Fusion](https://mlsanity.com/paper/2011.13090/) (2020)
- [Analysis of Multilingual Sequence-to-Sequence speech recognition systems](https://mlsanity.com/paper/1811.03451/) (2018)
- [Low-Rank Matrix Completion using Nuclear Norm with Facial Reduction](https://mlsanity.com/paper/1608.04168/) (2016)

## Abstract (arXiv)

Low resource automatic speech recognition (ASR) is a useful but thorny task, since deep learning ASR models usually need huge amounts of training data. The existing models mostly established a bottleneck (BN) layer by pre-training on a large source language, and transferring to the low resource target language. In this work, we introduced an adaptive activation network to the upper layers of ASR model, and applied different activation functions to different languages. We also proposed two approaches to train the model: (1) cross-lingual learning, replacing the activation function from source language to target language, (2) multilingual learning, jointly training the Connectionist Temporal Classification (CTC) loss of each language and the relevance of different languages. Our experiments on IARPA Babel datasets demonstrated that our approaches outperform the from-scratch training and traditional bottleneck feature based methods. In addition, combining the cross-lingual learning and multilingual learning together could further improve the performance of multilingual speech recognition.