# Connecting and Comparing Language Model Interpolation Techniques

arXiv: https://arxiv.org/abs/1908.09738 · HTML: https://mlsanity.com/paper/1908.09738/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Aug 2019.

## Built on by (most cited first, 10 of 10)

- [Synthetic Query Generation using Large Language Models for Virtual Assistants](https://mlsanity.com/paper/2406.06729/) (2024)
- [Error-driven Pruning of Language Models for Virtual Assistants](https://mlsanity.com/paper/2102.07219/) (2021)
- [Space-Efficient Representation of Entity-centric Query Language Models](https://mlsanity.com/paper/2206.14885/) (2022)
- [Modeling Spoken Information Queries for Virtual Assistants: Open Problems, Challenges and Opportunities](https://mlsanity.com/paper/2304.13149/) (2023)
- [Application-Agnostic Language Modeling for On-Device ASR](https://mlsanity.com/paper/2305.09764/) (2023)
- [Transformer-based Model for ASR N-Best Rescoring and Rewriting](https://mlsanity.com/paper/2406.08207/) (2024)
- [Towards Continual Entity Learning in Language Models for Conversational Agents](https://mlsanity.com/paper/2108.00082/) (2021)
- [Combining Language Models For Specialized Domains: A Colorful Approach](https://mlsanity.com/paper/2310.19708/) (2023)
- [Initial Decoding with Minimally Augmented Language Model for Improved Lattice Rescoring in Low Resource ASR](https://mlsanity.com/paper/2403.10937/) (2024)
- [Phonetically-Augmented Discriminative Rescoring for Voice Search Error Correction](https://mlsanity.com/paper/2506.06117/) (2025)

## Abstract (arXiv)

In this work, we uncover a theoretical connection between two language model interpolation techniques, count merging and Bayesian interpolation. We compare these techniques as well as linear interpolation in three scenarios with abundant training data per component model. Consistent with prior work, we show that both count merging and Bayesian interpolation outperform linear interpolation. We include the first (to our knowledge) published comparison of count merging and Bayesian interpolation, showing that the two techniques perform similarly. Finally, we argue that other considerations will make Bayesian interpolation the preferred approach in most circumstances.