# BaLEEN: Biasing with Latent Encoded Entities for Context-Aware ASR

arXiv: https://arxiv.org/abs/2609.36913 · HTML: https://mlsanity.com/paper/2609.36913/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 28 references that are in the map.
- First posted on arXiv 29 Sep 2026.

## Builds on (most cited first, 23 of 23)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Perceiver: General Perception with Iterative Attention](https://mlsanity.com/paper/2103.03206/) (2021)
- [HyperNetworks](https://mlsanity.com/paper/1609.09106/) (2016)
- [ByT5: Towards a token-free future with pre-trained byte-to-byte models](https://mlsanity.com/paper/2105.13626/) (2021)
- [Deep context: end-to-end contextual speech recognition](https://mlsanity.com/paper/1808.02480/) (2018)
- [Fast Conformer with Linearly Scalable Attention for Efficient Speech Recognition](https://mlsanity.com/paper/2305.05084/) (2023)
- [Contextualized Streaming End-to-End Speech Recognition with Trie-Based Deep Biasing and Shallow Fusion](https://mlsanity.com/paper/2104.02194/) (2021)
- [Contextual Adapters for Personalized Speech Recognition in Neural Transducers](https://mlsanity.com/paper/2205.13660/) (2022)
- [Context-Aware Transformer Transducer for Speech Recognition](https://mlsanity.com/paper/2111.03250/) (2021)
- [Contextualized End-to-End Speech Recognition with Contextual Phrase Prediction Network](https://mlsanity.com/paper/2305.12493/) (2023)
- [Dialog-context aware end-to-end speech recognition](https://mlsanity.com/paper/1808.02171/) (2018)
- [Fast Contextual Adaptation with Neural Associative Memory for On-Device Personalized Speech Recognition](https://mlsanity.com/paper/2110.02220/) (2021)
- [Contextualized End-to-end Automatic Speech Recognition with Intermediate Biasing Loss](https://mlsanity.com/paper/2406.16120/) (2024)
- [End-to-end contextual asr based on posterior distribution adaptation for hybrid ctc/attention system](https://mlsanity.com/paper/2202.09003/) (2022)
- [Fast Context-Biasing for CTC and Transducer ASR models with CTC-based Word Spotter](https://mlsanity.com/paper/2406.07096/) (2024)
- [Deferred NAM: Low-latency Top-K Context Injection via Deferred Context Encoding for Non-Streaming ASR](https://mlsanity.com/paper/2404.10180/) (2024)
- [InterBiasing: Boost Unseen Word Recognition through Biasing Intermediate Predictions](https://mlsanity.com/paper/2406.14890/) (2024)
- [Spike-Triggered Contextual Biasing for End-to-End Mandarin Speech Recognition](https://mlsanity.com/paper/2310.04657/) (2023)
- [WCTC-Biasing: Retraining-free Contextual Biasing ASR with Wildcard CTC-based Keyword Spotting and Inter-layer Biasing](https://mlsanity.com/paper/2506.01263/) (2025)
- [Zero-shot Context Biasing with Trie-based Decoding using Synthetic Multi-Pronunciation](https://mlsanity.com/paper/2508.17796/) (2025)
- [Contextualized Automatic Speech Recognition with Dynamic Vocabulary Prediction and Activation](https://mlsanity.com/paper/2505.23077/) (2025)

## Abstract (arXiv)

Transcribing domain-specific entities and rare proper nouns remains a major challenge in automatic speech recognition (ASR). In this paper, we propose BaLEEN (Biasing with Latent Encoded Entities), a lightweight, hypernetwork-based framework for dynamic contextual adaptation without fine-tuning the underlying ASR model. BaLEEN encodes variable-length contextual keywords using a pretrained language model, compresses them into a fixed sequence of latent vectors via a Perceiver bottleneck, and injects context-dependent bias vectors directly into the intermediate encoder representations of the ASR model. Because both the language model and the backbone ASR model remain entirely frozen during training, BaLEEN operates as a plug-and-play adapter that incurs zero computational overhead at inference time when context biases are precomputed. We evaluate our method on a CTC-based ASR model using a Wikipedia-derived corpus with annotated named entities and synthetic speech. Experimental results demonstrate that BaLEEN reduces keyword miss rate by 8.7% on the test set relative to the unbiased baseline while simultaneously improving overall word error rate by 21% and character error rate by 28%.