# ATCSpeechNet: A multilingual end-to-end speech recognition framework for air traffic control systems

arXiv: https://arxiv.org/abs/2102.08535 · HTML: https://mlsanity.com/paper/2102.08535/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 64 references that are in the map.
- First posted on arXiv 17 Feb 2021.

## Builds on (most cited first, 21 of 21)

- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Speech Recognition with Deep Recurrent Neural Networks](https://mlsanity.com/paper/1303.5778/) (2013)
- [Listen, Attend and Spell](https://mlsanity.com/paper/1508.01211/) (2015)
- [Deep Speech 2: End-to-End Speech Recognition in English and Mandarin](https://mlsanity.com/paper/1512.02595/) (2015)
- [AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baseline](https://mlsanity.com/paper/1709.05522/) (2017)
- [AISHELL-2: Transforming Mandarin ASR Research Into Industrial Scale](https://mlsanity.com/paper/1808.10583/) (2018)
- [Multilingual Speech Recognition With A Single End-To-End Model](https://mlsanity.com/paper/1711.01694/) (2017)
- [Jasper: An End-to-End Convolutional Neural Acoustic Model](https://mlsanity.com/paper/1904.03288/) (2019)
- [Bytes are All You Need: End-to-End Multilingual Speech Recognition and Synthesis with Bytes](https://mlsanity.com/paper/1811.09021/) (2018)
- [Very Deep Self-Attention Networks for End-to-End Speech Recognition](https://mlsanity.com/paper/1904.13377/) (2019)
- [Wav2Letter: an End-to-End ConvNet-based Speech Recognition System](https://mlsanity.com/paper/1609.03193/) (2016)
- [wav2letter++: The Fastest Open-source Speech Recognition System](https://mlsanity.com/paper/1812.07625/) (2018)
- [On the End-to-End Solution to Mandarin-English Code-switching Speech Recognition](https://mlsanity.com/paper/1811.00241/) (2018)
- [Fully Convolutional Speech Recognition](https://mlsanity.com/paper/1812.06864/) (2018)
- [Comparison of Decoding Strategies for CTC Acoustic Models](https://mlsanity.com/paper/1708.04469/) (2017)
- [The Airbus Air Traffic Control speech recognition 2018 challenge: towards ATC automatic transcription and call sign detection](https://mlsanity.com/paper/1810.12614/) (2018)
- [Feedforward Sequential Memory Networks: A New Structure to Learn Long-term Dependency](https://mlsanity.com/paper/1512.08301/) (2015)
- [ATCSpeech: a multilingual pilot-controller speech corpus from real Air Traffic Control environment](https://mlsanity.com/paper/1911.11365/) (2019)
- [A Comparison of Discrete Latent Variable Models for Speech Representation Learning](https://mlsanity.com/paper/2010.14230/) (2020)
- [Phoneme-Based Contextualization for Cross-Lingual Speech Recognition in End-to-End Models](https://mlsanity.com/paper/1906.09292/) (2019)
- [Improving speech recognition models with small samples for air traffic control systems](https://mlsanity.com/paper/2102.08015/) (2021)

## Built on by (most cited first, 6 of 6)

- [A Comparative Study of Speaker Role Identification in Air Traffic Communication Using Deep Learning Approaches](https://mlsanity.com/paper/2111.02041/) (2021)
- [Integrating spoken instructions into flight trajectory prediction to optimize automation in air traffic control](https://mlsanity.com/paper/2305.01661/) (2023)
- [ROSE: A Recognition-Oriented Speech Enhancement Framework in Air Traffic Control Using Multi-Objective Learning](https://mlsanity.com/paper/2312.06118/) (2023)
- [Towards certifiable AI in aviation: landscape, challenges, and opportunities](https://mlsanity.com/paper/2409.08666/) (2024)
- [Advancing Airport Tower Command Recognition: Integrating Squeeze-and-Excitation and Broadcasted Residual Learning](https://mlsanity.com/paper/2406.18313/) (2024)
- [SCOPE: A Lightweight-training LLM Framework for Air Traffic Control Readback Monitoring](https://mlsanity.com/paper/2605.29543/) (2026)

## Abstract (arXiv)

In this paper, a multilingual end-to-end framework, called as ATCSpeechNet, is proposed to tackle the issue of translating communication speech into human-readable text in air traffic control (ATC) systems. In the proposed framework, we focus on integrating the multilingual automatic speech recognition (ASR) into one model, in which an end-to-end paradigm is developed to convert speech waveform into text directly, without any feature engineering or lexicon. In order to make up for the deficiency of the handcrafted feature engineering caused by ATC challenges, a speech representation learning (SRL) network is proposed to capture robust and discriminative speech representations from the raw wave. The self-supervised training strategy is adopted to optimize the SRL network from unlabeled data, and further to predict the speech features, i.e., wave-to-feature. An end-to-end architecture is improved to complete the ASR task, in which a grapheme-based modeling unit is applied to address the multilingual ASR issue. Facing the problem of small transcribed samples in the ATC domain, an unsupervised approach with mask prediction is applied to pre-train the backbone network of the ASR model on unlabeled data by a feature-to-feature process. Finally, by integrating the SRL with ASR, an end-to-end multilingual ASR framework is formulated in a supervised manner, which is able to translate the raw wave into text in one model, i.e., wave-to-text. Experimental results on the ATCSpeech corpus demonstrate that the proposed approach achieves a high performance with a very small labeled corpus and less resource consumption, only 4.20% label error rate on the 58-hour transcribed corpus. Compared to the baseline model, the proposed approach obtains over 100% relative performance improvement which can be further enhanced with the increasing of the size of the transcribed samples.