# Optimal estimation of sparse topic models

arXiv: https://arxiv.org/abs/2001.07861 · HTML: https://mlsanity.com/paper/2001.07861/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Jan 2020.

## Built on by (most cited first, 5 of 5)

- [Unique sparse decomposition of low rank matrices](https://mlsanity.com/paper/2106.07736/) (2021)
- [Estimation and inference for the Wasserstein distance between mixing measures in topic models](https://mlsanity.com/paper/2206.12768/) (2022)
- [Likelihood estimation of sparse topic distributions in topic models and its applications to Wasserstein document distance calculations](https://mlsanity.com/paper/2107.05766/) (2021)
- [Beyond identifiability: Learning causal representations with few environments and finite samples](https://mlsanity.com/paper/2603.25796/) (2026)
- [Scalable Minimum-Volume Simplex Estimation with Non-asymptotic Analysis](https://mlsanity.com/paper/2609.25576/) (2026)

## Abstract (arXiv)

Topic models have become popular tools for dimension reduction and exploratory analysis of text data which consists in observed frequencies of a vocabulary of p words in n documents, stored in a p× n matrix. The main premise is that the mean of this data matrix can be factorized into a product of two non-negative matrices: a p× K word-topic matrix A and a K× n topic-document matrix W. This paper studies the estimation of A that is possibly element-wise sparse, and the number of topics K is unknown. In this under-explored context, we derive a new minimax lower bound for the estimation of such A and propose a new computationally efficient algorithm for its recovery. We derive a finite sample upper bound for our estimator, and show that it matches the minimax lower bound in many scenarios. Our estimate adapts to the unknown sparsity of A and our analysis is valid for any finite n, p, K and document lengths. Empirical results on both synthetic data and semi-synthetic data show that our proposed estimator is a strong competitor of the existing state-of-the-art algorithms for both non-sparse A and sparse A, and has superior performance is many scenarios of interest.