# Frequency of Frequencies Distributions and Size Dependent Exchangeable Random Partitions

arXiv: https://arxiv.org/abs/1608.00264 · HTML: https://mlsanity.com/paper/1608.00264/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Jul 2016.

## Built on by (most cited first, 2 of 2)

- [A Bayesian nonparametric approach to count-min sketch under power-law data streams](https://mlsanity.com/paper/2102.03743/) (2021)
- [Poisson Hierarchical Indian Buffet Processes-With Indications for Microbiome Species Sampling Models](https://mlsanity.com/paper/2502.01919/) (2025)

## Abstract (arXiv)

Motivated by the fundamental problem of modeling the frequency of frequencies (FoF) distribution, this paper introduces the concept of a cluster structure to define a probability function that governs the joint distribution of a random count and its exchangeable random partitions. A cluster structure, naturally arising from a completely random measure mixed Poisson process, allows the probability distribution of the random partitions of a subset of a population to be dependent on the population size, a distinct and motivated feature that makes it more flexible than a partition structure. This allows it to model an entire FoF distribution whose structural properties change as the population size varies. A FoF vector can be simulated by drawing an infinite number of Poisson random variables, or by a stick-breaking construction with a finite random number of steps. A generalized negative binomial process model is proposed to generate a cluster structure, where in the prior the number of clusters is finite and Poisson distributed, and the cluster sizes follow a truncated negative binomial distribution. We propose a simple Gibbs sampling algorithm to extrapolate the FoF vector of a population given the FoF vector of a sample taken without replacement from the population. We illustrate our results and demonstrate the advantages of the proposed models through the analysis of real text, genomic, and survey data.