# Text-Only Domain Adaptation for End-to-End Speech Recognition through Down-Sampling Acoustic Representation

arXiv: https://arxiv.org/abs/2309.02459 · HTML: https://mlsanity.com/paper/2309.02459/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 30 references that are in the map.
- First posted on arXiv 4 Sep 2023.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [ESPnet: End-to-End Speech Processing Toolkit](https://mlsanity.com/paper/1804.00015/) (2018)
- [Listen, Attend and Spell](https://mlsanity.com/paper/1508.01211/) (2015)
- [AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baseline](https://mlsanity.com/paper/1709.05522/) (2017)
- [WenetSpeech: A 10000+ Hours Multi-domain Mandarin Corpus for Speech Recognition](https://mlsanity.com/paper/2110.03370/) (2021)
- [Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognition](https://mlsanity.com/paper/2206.08317/) (2022)
- [AISHELL-2: Transforming Mandarin ASR Research Into Industrial Scale](https://mlsanity.com/paper/1808.10583/) (2018)
- [WeNet: Production oriented Streaming and Non-streaming End-to-End Speech Recognition Toolkit](https://mlsanity.com/paper/2102.01547/) (2021)
- [An analysis of incorporating an external language model into a sequence-to-sequence model](https://mlsanity.com/paper/1712.01996/) (2017)
- [Branchformer: Parallel MLP-Attention Architectures to Capture Local and Global Context for Speech Recognition and Understanding](https://mlsanity.com/paper/2207.02971/) (2022)
- [Internal Language Model Estimation for Domain-Adaptive End-to-End Speech Recognition](https://mlsanity.com/paper/2011.01991/) (2020)
- [MAESTRO: Matched Speech Text Representations through Modality Matching](https://mlsanity.com/paper/2204.03409/) (2022)
- [CIF: Continuous Integrate-and-Fire for End-to-End Speech Recognition](https://mlsanity.com/paper/1905.11235/) (2019)
- [End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Results](https://mlsanity.com/paper/1412.1602/) (2014)
- [Internal Language Model Training for Domain-Adaptive End-to-End Speech Recognition](https://mlsanity.com/paper/2102.01380/) (2021)
- [Investigating Methods to Improve Language Model Integration for Attention-based Encoder-Decoder ASR Models](https://mlsanity.com/paper/2104.05544/) (2021)
- [Injecting Text in Self-Supervised Speech Pretraining](https://mlsanity.com/paper/2108.12226/) (2021)
- [JOIST: A Joint Speech and Text Streaming Model For ASR](https://mlsanity.com/paper/2210.07353/) (2022)
- [Towards Reducing the Need for Speech Training Data To Build Spoken Language Understanding Systems](https://mlsanity.com/paper/2203.00006/) (2022)
- [Multitask Training with Text Data for End-to-End Speech Recognition](https://mlsanity.com/paper/2010.14318/) (2020)
- [Improving Hybrid CTC/Attention End-to-end Speech Recognition with Pretrained Acoustic and Language Model](https://mlsanity.com/paper/2112.07254/) (2021)
- [Exploring RNN-Transducer for Chinese Speech Recognition](https://mlsanity.com/paper/1811.05097/) (2018)
- [Improving Mandarin Speech Recogntion with Block-augmented Transformer](https://mlsanity.com/paper/2207.11697/) (2022)
- [Joint Audio/Text Training for Transformer Rescorer of Streaming Speech Recognition](https://mlsanity.com/paper/2211.00174/) (2022)

## Abstract (arXiv)

Mapping two modalities, speech and text, into a shared representation space, is a research topic of using text-only data to improve end-to-end automatic speech recognition (ASR) performance in new domains. However, the length of speech representation and text representation is inconsistent. Although the previous method up-samples the text representation to align with acoustic modality, it may not match the expected actual duration. In this paper, we proposed novel representations match strategy through down-sampling acoustic representation to align with text modality. By introducing a continuous integrate-and-fire (CIF) module generating acoustic representations consistent with token length, our ASR model can learn unified representations from both modalities better, allowing for domain adaptation using text-only data of the target domain. Experiment results of new domain data demonstrate the effectiveness of the proposed method.