# Unified Modeling of Multi-Domain Multi-Device ASR Systems

arXiv: https://arxiv.org/abs/2205.06655 · HTML: https://mlsanity.com/paper/2205.06655/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 22 references that are in the map.
- First posted on arXiv 13 May 2022.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Scaling Vision with Sparse Mixture of Experts](https://mlsanity.com/paper/2106.05974/) (2021)
- [DEMix Layers: Disentangling Domains for Modular Language Modeling](https://mlsanity.com/paper/2108.05036/) (2021)
- [Contextual RNN-T For Open Domain ASR](https://mlsanity.com/paper/2006.03411/) (2020)
- [Attention based on-device streaming speech recognition with large speech corpus](https://mlsanity.com/paper/2001.00577/) (2020)
- [REDAT: Accent-Invariant Representation for End-to-End ASR by Domain Adversarial Training with Relabeling](https://mlsanity.com/paper/2012.07353/) (2020)
- [Building a Unified Code-Switching ASR System for South African Languages](https://mlsanity.com/paper/1807.10949/) (2018)
- [Semi-supervised acoustic model training for five-lingual code-switched ASR](https://mlsanity.com/paper/1906.08647/) (2019)
- [Listen with Intent: Improving Speech Recognition with Audio-to-Intent Front-End](https://mlsanity.com/paper/2105.07071/) (2021)
- [AD3: Attentive Deep Document Dater](https://mlsanity.com/paper/1902.02161/) (2019)
- [Improving RNN-T ASR Performance with Date-Time and Location Awareness](https://mlsanity.com/paper/2106.06183/) (2021)
- [A Mixture of Expert Based Deep Neural Network for Improved ASR](https://mlsanity.com/paper/2112.01025/) (2021)

## Built on by (most cited first, 3 of 3)

- [Task Oriented Dialogue as a Catalyst for Self-Supervised Automatic Speech Recognition](https://mlsanity.com/paper/2401.02417/) (2024)
- [Multi-Stage Multi-Modal Pre-Training for Automatic Speech Recognition](https://mlsanity.com/paper/2403.19822/) (2024)
- [An Efficient Self-Learning Framework For Interactive Spoken Dialog Systems](https://mlsanity.com/paper/2409.10515/) (2024)

## Abstract (arXiv)

Modern Automatic Speech Recognition (ASR) systems often use a portfolio of domain-specific models in order to get high accuracy for distinct user utterance types across different devices. In this paper, we propose an innovative approach that integrates the different per-domain per-device models into a unified model, using a combination of domain embedding, domain experts, mixture of experts and adversarial training. We run careful ablation studies to show the benefit of each of these innovations in contributing to the accuracy of the overall unified model. Experiments show that our proposed unified modeling approach actually outperforms the carefully tuned per-domain models, giving relative gains of up to 10% over a baseline model with negligible increase in the number of parameters.