# Bag of biterms modeling for short texts

arXiv: https://arxiv.org/abs/2003.11948 · HTML: https://mlsanity.com/paper/2003.11948/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Mar 2020.

## Built on by (most cited first, 4 of 4)

- [NeuroMax: Enhancing Neural Topic Modeling via Maximizing Mutual Information and Group Topic Regularization](https://mlsanity.com/paper/2409.19749/) (2024)
- [GloCOM: A Short Text Neural Topic Model via Global Clustering Context](https://mlsanity.com/paper/2412.00525/) (2024)
- [XTRA: Cross-Lingual Topic Modeling with Topic and Representation Alignments](https://mlsanity.com/paper/2510.02788/) (2025)
- [CATCH: A Controllable Theme Detection Framework with Contextualized Clustering and Hierarchical Generation](https://mlsanity.com/paper/2512.21715/) (2025)

## Abstract (arXiv)

Analyzing texts from social media encounters many challenges due to their unique characteristics of shortness, massiveness, and dynamic. Short texts do not provide enough context information, causing the failure of the traditional statistical models. Furthermore, many applications often face with massive and dynamic short texts, causing various computational challenges to the current batch learning algorithms. This paper presents a novel framework, namely Bag of Biterms Modeling (BBM), for modeling massive, dynamic, and short text collections. BBM comprises of two main ingredients: (1) the concept of Bag of Biterms (BoB) for representing documents, and (2) a simple way to help statistical models to include BoB. Our framework can be easily deployed for a large class of probabilistic models, and we demonstrate its usefulness with two well-known models: Latent Dirichlet Allocation (LDA) and Hierarchical Dirichlet Process (HDP). By exploiting both terms (words) and biterms (pairs of words), the major advantages of BBM are: (1) it enhances the length of the documents and makes the context more coherent by emphasizing the word connotation and co-occurrence via Bag of Biterms, (2) it inherits inference and learning algorithms from the primitive to make it straightforward to design online and streaming algorithms for short texts. Extensive experiments suggest that BBM outperforms several state-of-the-art models. We also point out that the BoB representation performs better than the traditional representations (e.g, Bag of Words, tf-idf) even for normal texts.