# On-the-Fly Rectification for Robust Large-Vocabulary Topic Inference

arXiv: https://arxiv.org/abs/2111.06580 · HTML: https://mlsanity.com/paper/2111.06580/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 43 references that are in the map.
- First posted on arXiv 12 Nov 2021.

## Builds on (most cited first, 7 of 7)

- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Inertial Proximal Alternating Linearized Minimization (iPALM) for Nonconvex and Nonsmooth Problems](https://mlsanity.com/paper/1702.02505/) (2017)
- [Anchor-Free Correlated Topic Modeling: Identifiability and Algorithm](https://mlsanity.com/paper/1611.05010/) (2016)
- [Provable Algorithms for Inference in Topic Models](https://mlsanity.com/paper/1605.08491/) (2016)
- [A provable SVD-based algorithm for learning topics in dominant admixture corpus](https://mlsanity.com/paper/1410.6991/) (2014)
- [Low-dimensional Embeddings for Interpretable Anchor-based Topic Inference](https://mlsanity.com/paper/1711.06826/) (2017)
- [Robust Spectral Inference for Joint Stochastic Matrix Factorization](https://mlsanity.com/paper/1611.00175/) (2016)

## Abstract (arXiv)

Across many data domains, co-occurrence statistics about the joint appearance of objects are powerfully informative. By transforming unsupervised learning problems into decompositions of co-occurrence statistics, spectral algorithms provide transparent and efficient algorithms for posterior inference such as latent topic analysis and community detection. As object vocabularies grow, however, it becomes rapidly more expensive to store and run inference algorithms on co-occurrence statistics. Rectifying co-occurrence, the key process to uphold model assumptions, becomes increasingly more vital in the presence of rare terms, but current techniques cannot scale to large vocabularies. We propose novel methods that simultaneously compress and rectify co-occurrence statistics, scaling gracefully with the size of vocabulary and the dimension of latent space. We also present new algorithms learning latent variables from the compressed statistics, and verify that our methods perform comparably to previous approaches on both textual and non-textual data.