# The Microsoft 2017 Conversational Speech Recognition System

arXiv: https://arxiv.org/abs/1708.06073 · HTML: https://mlsanity.com/paper/1708.06073/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Aug 2017.

## Built on by (most cited first, 30 of 52)

- [AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecasting](https://mlsanity.com/paper/2103.14023/) (2021)
- [Dive into Deep Learning](https://mlsanity.com/paper/2106.11342/) (2021)
- [Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications](https://mlsanity.com/paper/2204.11786/) (2022)
- [On the limit of English conversational speech recognition](https://mlsanity.com/paper/2105.00982/) (2021)
- [Investigation of Data Augmentation Techniques for Disordered Speech Recognition](https://mlsanity.com/paper/2201.05562/) (2022)
- [Neural Delay Differential Equations](https://mlsanity.com/paper/2102.10801/) (2021)
- [FiT: Parameter Efficient Few-shot Transfer Learning for Personalized and Federated Image Classification](https://mlsanity.com/paper/2206.08671/) (2022)
- [Look Who's Talking: Active Speaker Detection in the Wild](https://mlsanity.com/paper/2108.07640/) (2021)
- [Audio-driven Neural Gesture Reenactment with Video Motion Graphs](https://mlsanity.com/paper/2207.11524/) (2022)
- [Recent Progress in the CUHK Dysarthric Speech Recognition System](https://mlsanity.com/paper/2201.05845/) (2022)
- [Transformer Language Models with LSTM-based Cross-utterance Information Representation](https://mlsanity.com/paper/2102.06474/) (2021)
- [Measuring the Accuracy of Automatic Speech Recognition Solutions](https://mlsanity.com/paper/2408.16287/) (2024)
- [Adapting Long Context NLM for ASR Rescoring in Conversational Agents](https://mlsanity.com/paper/2104.11070/) (2021)
- [Input Length Matters: Improving RNN-T and MWER Training for Long-form Telephony Speech Recognition](https://mlsanity.com/paper/2110.03841/) (2021)
- [Assessing SATNet's Ability to Solve the Symbol Grounding Problem](https://mlsanity.com/paper/2312.11522/) (2023)
- [Platform for Situated Intelligence](https://mlsanity.com/paper/2103.15975/) (2021)
- [Speech-Mamba: Long-Context Speech Recognition with Selective State Spaces Models](https://mlsanity.com/paper/2409.18654/) (2024)
- [On Architectures and Training for Raw Waveform Feature Extraction in ASR](https://mlsanity.com/paper/2104.04298/) (2021)
- [ASR Adaptation for E-commerce Chatbots using Cross-Utterance Context and Multi-Task Language Modeling](https://mlsanity.com/paper/2106.09532/) (2021)
- [Language technology practitioners as language managers: arbitrating data bias and predictive bias in ASR](https://mlsanity.com/paper/2202.12603/) (2022)
- [Modular Domain Adaptation for Conformer-Based Streaming ASR](https://mlsanity.com/paper/2305.13408/) (2023)
- [Multi-Encoder Learning and Stream Fusion for Transformer-Based End-to-End Automatic Speech Recognition](https://mlsanity.com/paper/2104.00120/) (2021)
- [ResNEsts and DenseNEsts: Block-based DNN Models with Improved Representation Guarantees](https://mlsanity.com/paper/2111.05496/) (2021)
- [Contextual Squeeze-and-Excitation for Efficient Few-Shot Image Classification](https://mlsanity.com/paper/2206.09843/) (2022)
- [LiveSeg: Unsupervised Multimodal Temporal Segmentation of Long Livestream Videos](https://mlsanity.com/paper/2210.05840/) (2022)
- [Personalized Predictive ASR for Latency Reduction in Voice Assistants](https://mlsanity.com/paper/2305.13794/) (2023)
- [SAPIEN: Affective Virtual Agents Powered by Large Language Models](https://mlsanity.com/paper/2308.03022/) (2023)
- [Role of Data Augmentation Strategies in Knowledge Distillation for Wearable Sensor Data](https://mlsanity.com/paper/2201.00111/) (2022)
- [MyMove: Facilitating Older Adults to Collect In-Situ Activity Labels on a Smartwatch with Speech](https://mlsanity.com/paper/2204.00145/) (2022)
- [Large-Scale News Classification using BERT Language Model: Spark NLP Approach](https://mlsanity.com/paper/2107.06785/) (2021)

## Abstract (arXiv)

We describe the 2017 version of Microsoft's conversational speech recognition system, in which we update our 2016 system with recent developments in neural-network-based acoustic and language modeling to further advance the state of the art on the Switchboard speech recognition task. The system adds a CNN-BLSTM acoustic model to the set of model architectures we combined previously, and includes character-based and dialog session aware LSTM language models in rescoring. For system combination we adopt a two-stage approach, whereby subsets of acoustic models are first combined at the senone/frame level, followed by a word-level voting via confusion networks. We also added a confusion network rescoring step after system combination. The resulting system yields a 5.1% word error rate on the 2000 Switchboard evaluation set.