# Contextualized Token Discrimination for Speech Search Query Correction

arXiv: https://arxiv.org/abs/2509.04393 · HTML: https://mlsanity.com/paper/2509.04393/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 62 references that are in the map.
- First posted on arXiv 4 Sep 2025.

## Builds on (most cited first, 30 of 34)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour](https://mlsanity.com/paper/1706.02677/) (2017)
- [How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings](https://mlsanity.com/paper/1909.00512/) (2019)
- [AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baseline](https://mlsanity.com/paper/1709.05522/) (2017)
- [Enhanced LSTM for Natural Language Inference](https://mlsanity.com/paper/1609.06038/) (2016)
- [Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition](https://mlsanity.com/paper/2010.10504/) (2020)
- [Deep context: end-to-end contextual speech recognition](https://mlsanity.com/paper/1808.02480/) (2018)
- [End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architectures](https://mlsanity.com/paper/1911.08460/) (2019)
- [Exploring Architectures, Data and Units For Streaming End-to-End Speech Recognition with RNN-Transducer](https://mlsanity.com/paper/1801.00841/) (2018)
- [Spelling Error Correction with Soft-Masked BERT](https://mlsanity.com/paper/2005.07421/) (2020)
- [The Past Mistake is the Future Wisdom: Error-driven Contrastive Probability Optimization for Chinese Spell Checking](https://mlsanity.com/paper/2203.00991/) (2022)
- [ASR Error Correction and Domain Adaptation Using Machine Translation](https://mlsanity.com/paper/2003.07692/) (2020)
- [Table Search Using a Deep Contextualized Language Model](https://mlsanity.com/paper/2005.09207/) (2020)
- [ESPnet-se: end-to-end speech enhancement and separation toolkit designed for asr integration](https://mlsanity.com/paper/2011.03706/) (2020)
- [Rethinking Masked Language Modeling for Chinese Spelling Correction](https://mlsanity.com/paper/2305.17721/) (2023)
- [Contextualized Embeddings in Named-Entity Recognition: An Empirical Study on Generalization](https://mlsanity.com/paper/2001.08053/) (2020)
- [Disentangled Phonetic Representation for Chinese Spelling Correction](https://mlsanity.com/paper/2305.14783/) (2023)
- [Dial-In LLM: Human-Aligned LLM-in-the-loop Intent Clustering for Customer Service Dialogues](https://mlsanity.com/paper/2412.09049/) (2024)
- [Content Selection Network for Document-grounded Retrieval-based Chatbots](https://mlsanity.com/paper/2101.08426/) (2021)
- [Familia: A Configurable Topic Modeling Framework for Industrial Text Engineering](https://mlsanity.com/paper/1808.03733/) (2018)
- [ASR-EC Benchmark: Evaluating Large Language Models on Chinese ASR Error Correction](https://mlsanity.com/paper/2412.03075/) (2024)
- [Contextual ASR Error Handling with LLMs Augmentation for Goal-Oriented Conversational AI](https://mlsanity.com/paper/2501.06129/) (2025)
- [QualBench: Benchmarking Chinese LLMs with Localized Professional Qualifications for Vertical Domain Evaluation](https://mlsanity.com/paper/2505.05225/) (2025)
- [Audio-to-score alignment of piano music using RNN-based automatic music transcription](https://mlsanity.com/paper/1711.04480/) (2017)

## Built on by (most cited first, 1 of 1)

- [Federated Heterogeneous Language Model Optimization for Hybrid Automatic Speech Recognition](https://mlsanity.com/paper/2603.04945/) (2026)

## Abstract (arXiv)

Query spelling correction is an important function of modern search engines since it effectively helps users express their intentions clearly. With the growing popularity of speech search driven by Automated Speech Recognition (ASR) systems, this paper introduces a novel method named Contextualized Token Discrimination (CTD) to conduct effective speech query correction. In CTD, we first employ BERT to generate token-level contextualized representations and then construct a composition layer to enhance semantic information. Finally, we produce the correct query according to the aggregated token representation, correcting the incorrect tokens by comparing the original token representations and the contextualized representations. Extensive experiments demonstrate the superior performance of our proposed method across all metrics, and we further present a new benchmark dataset with erroneous ASR transcriptions to offer comprehensive evaluations for audio query correction.