# Generalized Grade-of-Membership Estimation for High-dimensional Locally Dependent Data

arXiv: https://arxiv.org/abs/2412.19796 · HTML: https://mlsanity.com/paper/2412.19796/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 73 references that are in the map.
- First posted on arXiv 27 Dec 2024.

## Builds on (most cited first, 17 of 17)

- [Spectral Methods for Data Science: A Statistical Perspective](https://mlsanity.com/paper/2012.08496/) (2020)
- [Detecting Overlapping Communities in Networks Using Spectral Methods](https://mlsanity.com/paper/1412.3432/) (2014)
- [Estimating Mixed Memberships with Sharp Eigenvector Deviations](https://mlsanity.com/paper/1709.00407/) (2017)
- [Mixed Membership Estimation for Social Networks](https://mlsanity.com/paper/1708.07852/) (2017)
- [Inference for Heteroskedastic PCA with Missing Data](https://mlsanity.com/paper/2107.12365/) (2021)
- [The multilayer random dot product graph](https://mlsanity.com/paper/2007.10455/) (2020)
- [Using SVD for Topic Modeling](https://mlsanity.com/paper/1704.07016/) (2017)
- [Spectral embedding for dynamic networks with stability guarantees](https://mlsanity.com/paper/2106.01282/) (2021)
- [Entrywise Estimation of Singular Vectors of Low-Rank Matrices with Heteroskedasticity and Dependence](https://mlsanity.com/paper/2105.13346/) (2021)
- [Improvements on SCORE, Especially for Weak Signals](https://mlsanity.com/paper/1811.05927/) (2018)
- [SIMPLE: Statistical Inference on Membership Profiles in Large Networks](https://mlsanity.com/paper/1910.01734/) (2019)
- [Hypothesis Testing for Equality of Latent Positions in Random Graphs](https://mlsanity.com/paper/2105.10838/) (2021)
- [Intensity Profile Projection: A Framework for Continuous-Time Representation Learning for Dynamic Networks](https://mlsanity.com/paper/2306.06155/) (2023)
- [A Spectral Method for Identifiable Grade of Membership Analysis with Binary Responses](https://mlsanity.com/paper/2305.03149/) (2023)
- [Dimension-Grouped Mixed Membership Models for Multivariate Categorical Data](https://mlsanity.com/paper/2109.11705/) (2021)
- [Fast moment estimation for generalized latent Dirichlet models](https://mlsanity.com/paper/1603.05324/) (2016)
- [A Generalized Latent Factor Model Approach to Mixed-data Matrix Completion with Entrywise Consistency](https://mlsanity.com/paper/2211.09272/) (2022)

## Abstract (arXiv)

This work focuses on the mixed membership models for multivariate categorical data widely used for analyzing survey responses and population genetics data. These grade of membership (GoM) models offer rich modeling power but present significant estimation challenges for high-dimensional polytomous data. Popular existing approaches, such as Bayesian MCMC inference, are not scalable and lack theoretical guarantees in high-dimensional settings. To address this, we first observe that data from this model can be reformulated as a three-way (quasi-)tensor, with many subjects responding to many items with varying numbers of categories. We introduce a novel and simple approach that flattens the three-way quasi-tensor into a "fat" matrix, and then perform a singular value decomposition of it to estimate parameters by exploiting the singular subspace geometry. Our fast spectral method can accommodate a broad range of data distributions with arbitrarily locally dependent noise, which we formalize as the generalized-GoM models. We establish finite-sample entrywise error bounds for the generalized-GoM model parameters. This is supported by a new sharp two-to-infinity singular subspace perturbation theory for locally dependent and flexibly distributed noise, a contribution of independent interest. Simulations and applications to data in political surveys, population genetics, and single-cell sequencing demonstrate our method's superior performance.