# Attentive Temporal Pooling for Conformer-based Streaming Language Identification in Long-form Speech

arXiv: https://arxiv.org/abs/2202.12163 · HTML: https://mlsanity.com/paper/2202.12163/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 63 references that are in the map.
- First posted on arXiv 24 Feb 2022.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [Recent Developments on ESPnet Toolkit Boosted by Conformer](https://mlsanity.com/paper/2010.13956/) (2020)
- [VoxLingua107: a Dataset for Spoken Language Recognition](https://mlsanity.com/paper/2011.12998/) (2020)
- [Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling](https://mlsanity.com/paper/1902.08295/) (2019)
- [Recognizing long-form speech using streaming end-to-end models](https://mlsanity.com/paper/1910.11455/) (2019)
- [Automatic Language Identification in Texts: A Survey](https://mlsanity.com/paper/1804.08186/) (2018)
- [Optimizing Speech Recognition For The Edge](https://mlsanity.com/paper/1909.12408/) (2019)
- [On the End-to-End Solution to Mandarin-English Code-switching Speech Recognition](https://mlsanity.com/paper/1811.00241/) (2018)
- [Attention-Based Models for Text-Dependent Speaker Verification](https://mlsanity.com/paper/1710.10470/) (2017)
- [On the efficient representation and execution of deep acoustic models](https://mlsanity.com/paper/1607.04683/) (2016)
- [End-to-end Neural Diarization: From Transformer to Conformer](https://mlsanity.com/paper/2106.07167/) (2021)
- [A Conformer-based ASR Frontend for Joint Acoustic Echo Cancellation, Speech Enhancement and Speech Separation](https://mlsanity.com/paper/2111.09935/) (2021)
- [OLR 2021 Challenge: Datasets, Rules and Baselines](https://mlsanity.com/paper/2107.11113/) (2021)
- [Signal Combination for Language Identification](https://mlsanity.com/paper/1910.09687/) (2019)
- [Streaming Language Identification using Combination of Acoustic Representations and ASR Hypotheses](https://mlsanity.com/paper/2006.00703/) (2020)
- [Tuplemax Loss for Language Identification](https://mlsanity.com/paper/1811.12290/) (2018)
- [Cross-attention conformer for context modeling in speech enhancement for ASR](https://mlsanity.com/paper/2111.00127/) (2021)

## Built on by (most cited first, 6 of 6)

- [Miipher: A Robust Speech Restoration Model Integrating Self-Supervised Speech and Text Representations](https://mlsanity.com/paper/2303.01664/) (2023)
- [FLEURS-R: A Restored Multilingual Speech Corpus for Generation Tasks](https://mlsanity.com/paper/2408.06227/) (2024)
- [Highly Efficient Real-Time Streaming and Fully On-Device Speaker Diarization with Multi-Stage Clustering](https://mlsanity.com/paper/2210.13690/) (2022)
- [Parameter-Free Attentive Scoring for Speaker Verification](https://mlsanity.com/paper/2203.05642/) (2022)
- [Personalizing Keyword Spotting with Speaker Information](https://mlsanity.com/paper/2311.03419/) (2023)
- [Multimodal Modeling For Spoken Language Identification](https://mlsanity.com/paper/2309.10567/) (2023)

## Abstract (arXiv)

In this paper, we introduce a novel language identification system based on conformer layers. We propose an attentive temporal pooling mechanism to allow the model to carry information in long-form audio via a recurrent form, such that the inference can be performed in a streaming fashion. Additionally, we investigate two domain adaptation approaches to allow adapting an existing language identification model without retraining the model parameters for a new domain. We perform a comparative study of different model topologies under different constraints of model size, and find that conformer-based models significantly outperform LSTM and transformer based models. Our experiments also show that attentive temporal pooling and domain adaptation improve model accuracy.