# Large-scale entity resolution via microclustering Ewens--Pitman random partitions

arXiv: https://arxiv.org/abs/2507.18101 · HTML: https://mlsanity.com/paper/2507.18101/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 26 references that are in the map.
- First posted on arXiv 24 Jul 2025.

## Builds on (most cited first, 6 of 6)

- [An Alternative View: When Does SGD Escape Local Minima?](https://mlsanity.com/paper/1802.06175/) (2018)
- [Are Gibbs-type priors the most natural generalization of the Dirichlet process?](https://mlsanity.com/paper/1503.00163/) (2015)
- [Flexible Models for Microclustering with Application to Entity Resolution](https://mlsanity.com/paper/1610.09780/) (2016)
- [Random Partition Models for Microclustering Tasks](https://mlsanity.com/paper/2004.02008/) (2020)
- [Fast Learning of Clusters and Topics via Sparse Posteriors](https://mlsanity.com/paper/1609.07521/) (2016)
- [Non-exchangeable random partition models for microclustering](https://mlsanity.com/paper/1711.07287/) (2017)

## Abstract (arXiv)

We introduce the microclustering Ewens--Pitman model for random partitions, obtained by scaling the strength parameter of the Ewens--Pitman model linearly with the sample size. The resulting random partition is shown to have the microclustering property, namely: the size of the largest cluster grows sub-linearly with the sample size, while the number of clusters grows linearly. By leveraging the interplay between the Ewens--Pitman random partition with the Pitman--Yor process, we develop efficient variational inference schemes for posterior computation in entity resolution. Our approach achieves a speed-up of three orders of magnitude over existing Bayesian methods for entity resolution, while maintaining competitive empirical performance.