# Speech recognition for air traffic control via feature learning and end-to-end training

arXiv: https://arxiv.org/abs/2111.02654 · HTML: https://mlsanity.com/paper/2111.02654/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 25 references that are in the map.
- First posted on arXiv 4 Nov 2021.

## Builds on (most cited first, 14 of 14)

- [Deep Speech 2: End-to-End Speech Recognition in English and Mandarin](https://mlsanity.com/paper/1512.02595/) (2015)
- [Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learning](https://mlsanity.com/paper/1609.06773/) (2016)
- [Jasper: An End-to-End Convolutional Neural Acoustic Model](https://mlsanity.com/paper/1904.03288/) (2019)
- [wav2letter++: The Fastest Open-source Speech Recognition System](https://mlsanity.com/paper/1812.07625/) (2018)
- [Neural Speech Recognizer: Acoustic-to-Word LSTM Model for Large Vocabulary Speech Recognition](https://mlsanity.com/paper/1610.09975/) (2016)
- [Interpretable Convolutional Filters with SincNet](https://mlsanity.com/paper/1811.09725/) (2018)
- [Automatic Speech Recognition Benchmark for Air-Traffic Communications](https://mlsanity.com/paper/2006.10304/) (2020)
- [Efficiently Fusing Pretrained Acoustic and Linguistic Encoders for Low-resource Speech Recognition](https://mlsanity.com/paper/2101.06699/) (2021)
- [Contextual Semi-Supervised Learning: An Approach To Leverage Air-Surveillance and Untranscribed ATC Data in ASR Systems](https://mlsanity.com/paper/2104.03643/) (2021)
- [ATCSpeech: a multilingual pilot-controller speech corpus from real Air Traffic Control environment](https://mlsanity.com/paper/1911.11365/) (2019)
- [Improving speech recognition models with small samples for air traffic control systems](https://mlsanity.com/paper/2102.08015/) (2021)
- [Speech and Speaker Recognition from Raw Waveform with SincNet](https://mlsanity.com/paper/1812.05920/) (2018)
- [Lightweight End-to-End Speech Recognition from Raw Audio Data Using Sinc-Convolutions](https://mlsanity.com/paper/2010.07597/) (2020)
- [A Comparative Study of Speaker Role Identification in Air Traffic Communication Using Deep Learning Approaches](https://mlsanity.com/paper/2111.02041/) (2021)

## Built on by (most cited first, 2 of 2)

- [A Virtual Simulation-Pilot Agent for Training of Air Traffic Controllers](https://mlsanity.com/paper/2304.07842/) (2023)
- [Key Frame Mechanism For Efficient Conformer Based End-to-end Speech Recognition](https://mlsanity.com/paper/2310.14954/) (2023)

## Abstract (arXiv)

In this work, we propose a new automatic speech recognition (ASR) system based on feature learning and an end-to-end training procedure for air traffic control (ATC) systems. The proposed model integrates the feature learning block, recurrent neural network (RNN), and connectionist temporal classification loss to build an end-to-end ASR model. Facing the complex environments of ATC speech, instead of the handcrafted features, a learning block is designed to extract informative features from raw waveforms for acoustic modeling. Both the SincNet and 1D convolution blocks are applied to process the raw waveforms, whose outputs are concatenated to the RNN layers for the temporal modeling. Thanks to the ability to learn representations from raw waveforms, the proposed model can be optimized in a complete end-to-end manner, i.e., from waveform to text. Finally, the multilingual issue in the ATC domain is also considered to achieve the ASR task by constructing a combined vocabulary of Chinese characters and English letters. The proposed approach is validated on a multilingual real-world corpus (ATCSpeech), and the experimental results demonstrate that the proposed approach outperforms other baselines, achieving a 6.9% character error rate.