# Contextual Semi-Supervised Learning: An Approach To Leverage Air-Surveillance and Untranscribed ATC Data in ASR Systems

arXiv: https://arxiv.org/abs/2104.03643 · HTML: https://mlsanity.com/paper/2104.03643/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 34 references that are in the map.
- First posted on arXiv 8 Apr 2021.

## Builds on (most cited first, 3 of 3)

- [Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition](https://mlsanity.com/paper/2010.10504/) (2020)
- [Automatic Speech Recognition Benchmark for Air-Traffic Communications](https://mlsanity.com/paper/2006.10304/) (2020)
- [Exploiting semi-supervised training through a dropout regularization in end-to-end speech recognition](https://mlsanity.com/paper/1908.05227/) (2019)

## Built on by (most cited first, 12 of 12)

- [How Does Pre-trained Wav2Vec 2.0 Perform on Domain Shifted ASR? An Extensive Benchmark on Air Traffic Control Communications](https://mlsanity.com/paper/2203.16822/) (2022)
- [ATCO2 corpus: A Large-Scale Dataset for Research on Automatic Speech Recognition and Natural Language Understanding of Air Traffic Control Communications](https://mlsanity.com/paper/2211.04054/) (2022)
- [BERTraffic: BERT-based Joint Speaker Role and Speaker Change Detection for Air Traffic Control Communications](https://mlsanity.com/paper/2110.05781/) (2021)
- [Improving callsign recognition with air-surveillance data in air-traffic communication](https://mlsanity.com/paper/2108.12156/) (2021)
- [A two-step approach to leverage contextual data: speech recognition in air-traffic communications](https://mlsanity.com/paper/2202.03725/) (2022)
- [Implementing contextual biasing in GPU decoder for online ASR](https://mlsanity.com/paper/2306.15685/) (2023)
- [A Virtual Simulation-Pilot Agent for Training of Air Traffic Controllers](https://mlsanity.com/paper/2304.07842/) (2023)
- [Grammar Based Speaker Role Identification for Air Traffic Control Speech Recognition](https://mlsanity.com/paper/2108.12175/) (2021)
- [Speech recognition for air traffic control via feature learning and end-to-end training](https://mlsanity.com/paper/2111.02654/) (2021)
- [Speech and Natural Language Processing Technologies for Pseudo-Pilot Simulator](https://mlsanity.com/paper/2212.07164/) (2022)
- [Lessons Learned in ATCO2: 5000 hours of Air Traffic Control Communications for Robust Automatic Speech Recognition and Understanding](https://mlsanity.com/paper/2305.01155/) (2023)
- [Fast Streaming Transducer ASR Prototyping via Knowledge Distillation with Whisper](https://mlsanity.com/paper/2409.13499/) (2024)

## Abstract (arXiv)

Air traffic management and specifically air-traffic control (ATC) rely mostly on voice communications between Air Traffic Controllers (ATCos) and pilots. In most cases, these voice communications follow a well-defined grammar that could be leveraged in Automatic Speech Recognition (ASR) technologies. The callsign used to address an airplane is an essential part of all ATCo-pilot communications. We propose a two-steps approach to add contextual knowledge during semi-supervised training to reduce the ASR system error rates at recognizing the part of the utterance that contains the callsign. Initially, we represent in a WFST the contextual knowledge (i.e. air-surveillance data) of an ATCo-pilot communication. Then, during Semi-Supervised Learning (SSL) the contextual knowledge is added by second-pass decoding (i.e. lattice re-scoring). Results show that `unseen domains' (e.g. data from airports not present in the supervised training data) are further aided by contextual SSL when compared to standalone SSL. For this task, we introduce the Callsign Word Error Rate (CA-WER) as an evaluation metric, which only assesses ASR performance of the spoken callsign in an utterance. We obtained a 32.1% CA-WER relative improvement applying SSL with an additional 17.5% CA-WER improvement by adding contextual knowledge during SSL on a challenging ATC-based test set gathered from LiveATC.