# HK-LegiCoST: Leveraging Non-Verbatim Transcripts for Speech Translation

arXiv: https://arxiv.org/abs/2306.11252 · HTML: https://mlsanity.com/paper/2306.11252/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 30 references that are in the map.
- First posted on arXiv 20 Jun 2023.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019)
- [Language-agnostic BERT Sentence Embedding](https://mlsanity.com/paper/2007.01852/) (2020)
- [Beyond English-Centric Multilingual Machine Translation](https://mlsanity.com/paper/2010.11125/) (2020)
- [ESPnet: End-to-End Speech Processing Toolkit](https://mlsanity.com/paper/1804.00015/) (2018)
- [FLEURS: Few-shot Learning Evaluation of Universal Representations of Speech](https://mlsanity.com/paper/2205.12446/) (2022)
- [Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learning](https://mlsanity.com/paper/1609.06773/) (2016)
- [Europarl-ST: A Multilingual Corpus For Speech Translation Of Parliamentary Debates](https://mlsanity.com/paper/1911.03167/) (2019)
- [The Multilingual TEDx Corpus for Speech Recognition and Translation](https://mlsanity.com/paper/2102.01757/) (2021)
- [CVSS Corpus and Massively Multilingual Speech-to-Speech Translation](https://mlsanity.com/paper/2201.03713/) (2022)
- [Augmenting Librispeech with French Translations: A Multimodal Corpus for Direct Speech Translation Evaluation](https://mlsanity.com/paper/1802.03142/) (2018)
- [Speech-to-Speech Translation For A Real-world Unwritten Language](https://mlsanity.com/paper/2211.06474/) (2022)
- [The MGB-2 Challenge: Arabic Multi-Dialect Broadcast Media Recognition](https://mlsanity.com/paper/1609.05625/) (2016)
- [SDS-200: A Swiss German Speech to Standard German Text Corpus](https://mlsanity.com/paper/2205.09501/) (2022)
- [Automatic Speech Recognition Datasets in Cantonese: A Survey and New Dataset](https://mlsanity.com/paper/2201.02419/) (2022)

## Built on by (most cited first, 3 of 3)

- [Learning from Flawed Data: Weakly Supervised Automatic Speech Recognition](https://mlsanity.com/paper/2309.15796/) (2023)
- [Towards Comprehensive Semantic Speech Embeddings for Chinese Dialects](https://mlsanity.com/paper/2601.07274/) (2026)
- [Anchoring Speech with Semantics: A Multimodal Adapter Mechanism for Automatic Speech Recognition in Low-Resource Languages](https://mlsanity.com/paper/2608.29239/) (2026)

## Abstract (arXiv)

We introduce HK-LegiCoST, a new three-way parallel corpus of Cantonese-English translations, containing 600+ hours of Cantonese audio, its standard traditional Chinese transcript, and English translation, segmented and aligned at the sentence level. We describe the notable challenges in corpus preparation: segmentation, alignment of long audio recordings, and sentence-level alignment with non-verbatim transcripts. Such transcripts make the corpus suitable for speech translation research when there are significant differences between the spoken and written forms of the source language. Due to its large size, we are able to demonstrate competitive speech translation baselines on HK-LegiCoST and extend them to promising cross-corpus results on the FLEURS Cantonese subset. These results deliver insights into speech recognition and translation research in languages for which non-verbatim or ``noisy'' transcription is common due to various factors, including vernacular and dialectal speech.