mlsanityData to 7 Oct 2026

Lessons Learned in ATCO2: 5000 hours of Air Traffic Control Communications for Robust Automatic Speech Recognition and Understanding

arXiv 2305.01155 ↗First posted 2 May 2023eess.AS cs.CL cs.HCcited by 1

Not on the map. Why?

Abstract (arXiv)

Voice communication between air traffic controllers (ATCos) and pilots is critical for ensuring safe and efficient air traffic control (ATC). This task requires high levels of awareness from ATCos and can be tedious and error-prone. Recent attempts have been made to integrate artificial intelligence (AI) into ATC in order to reduce the workload of ATCos. However, the development of data-driven AI systems for ATC demands large-scale annotated datasets, which are currently lacking in the field. This paper explores the lessons learned from the ATCO2 project, a project that aimed to develop a unique platform to collect and preprocess large amounts of ATC data from airspace in real time. Audio and surveillance data were collected from publicly accessible radio frequency channels with VHF receivers owned by a community of volunteers and later uploaded to Opensky Network servers, which can be considered an "unlimited source" of data. In addition, this paper reviews previous work from ATCO2 partners, including (i) robust automatic speech recognition, (ii) natural language processing, (iii) English language identification of ATC communications, and (iv) the integration of surveillance data such as ADS-B. We believe that the pipeline developed during the ATCO2 project, along with the open-sourcing of its data, will encourage research in the ATC field. A sample of the ATCO2 corpus is available on the following website: https://www.atco2.org/data, while the full corpus can be purchased through ELDA at http://catalog.elra.info/en-us/repository/browse/ELRA-S0484. We demonstrated that ATCO2 is an appropriate dataset to develop ASR engines when little or near to no ATC in-domain data is available. For instance, with the CNN-TDNNf kaldi model, we reached the performance of as low as 17.9% and 24.9% WER on public ATC datasets which is 6.6/7.6% better than "out-of-domain" but supervised CNN-TDNNf model.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 32 of its 85 references that are in the map. First posted on arXiv 2 May 2023.

Builds on

32 of 85 references.

References · 32

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  5. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  6. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  7. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  8. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,358
  9. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,219
  10. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 589
  11. 2021Datasets: A Community Library for Natural Language Processingcited by 526
  12. 2015Attention-Based Models for Speech Recognitioncited by 400
20 more
  1. 2019vq-wav2vec: Self-Supervised Learning of Discrete Speech Representationscited by 335
  2. 2019A Survey on Recent Advances in Named Entity Recognition from Deep Learning modelscited by 117
  3. 2019End-to-End Neural Speaker Diarization with Permutation-Free Objectivescited by 86
  4. 2019End-to-End Neural Speaker Diarization with Self-attentioncited by 74
  5. 2020Bayesian HMM clustering of x-vector sequences (VBx) in speaker diarization: theory, implementation and analysis on standard taskscited by 69
  6. 2019The Second DIHARD Diarization Challenge: Dataset, task, and baselinescited by 39
  7. 2022How Does Pre-trained Wav2Vec 2.0 Perform on Domain Shifted ASR? An Extensive Benchmark on Air Traffic Control Communicationscited by 30
  8. 2018End-to-end contextual speech recognition using class language models and a token passing decodercited by 26
  9. 2022ATCO2 corpus: A Large-Scale Dataset for Research on Automatic Speech Recognition and Natural Language Understanding of Air Traffic Control Communicationscited by 19
  10. 2020Automatic Speech Recognition Benchmark for Air-Traffic Communicationscited by 17
  11. 2018The Airbus Air Traffic Control speech recognition 2018 challenge: towards ATC automatic transcription and call sign detectioncited by 13
  12. 2021BERTraffic: BERT-based Joint Speaker Role and Speaker Change Detection for Air Traffic Control Communicationscited by 13
  13. 2020A Survey on Recent Advances in Sequence Labeling from Deep Learning Modelscited by 12
  14. 2021Contextual Semi-Supervised Learning: An Approach To Leverage Air-Surveillance and Untranscribed ATC Data in ASR Systemscited by 12
  15. 2021Improving callsign recognition with air-surveillance data in air-traffic communicationcited by 11
  16. 2022A two-step approach to leverage contextual data: speech recognition in air-traffic communicationscited by 8
  17. 2020Speech Activity Detection Based on Multilingual Speech Recognition Systemcited by 4
  18. 2021Detecting English Speech in the Air Traffic Control Voice Communicationcited by 4
  19. 2023A Virtual Simulation-Pilot Agent for Training of Air Traffic Controllerscited by 3
  20. 2021Grammar Based Speaker Role Identification for Air Traffic Control Speech Recognitioncited by 2

Built on by

Cited by year

Citing papers per year: 2023: 1.

1 paper.

  1. 2023Implementing contextual biasing in GPU decoder for online ASRcited by 4