# Sparse PPMI Graph Averaging for Random Indexing Embeddings

arXiv: https://arxiv.org/abs/2608.05724 · HTML: https://mlsanity.com/paper/2608.05724/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 18 references that are in the map.
- First posted on arXiv 6 Aug 2026.

## Builds on (most cited first, 6 of 6)

- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [All-but-the-Top: Simple and Effective Postprocessing for Word Representations](https://mlsanity.com/paper/1702.01417/) (2017)
- [SimLex-999: Evaluating Semantic Models with (Genuine) Similarity Estimation](https://mlsanity.com/paper/1408.3456/) (2014)
- [Hash Embeddings for Efficient Word Representations](https://mlsanity.com/paper/1709.03933/) (2017)
- [Scalable MatMul-free Language Modeling](https://mlsanity.com/paper/2406.02528/) (2024)
- [Getting deep recommenders fit: Bloom embeddings for sparse binary input/output networks](https://mlsanity.com/paper/1706.03993/) (2017)

## Abstract (arXiv)

We study a specific sparse post-processing pipeline for Random Indexing (RI) on kinship analogies in a small fairytales corpus. The published artifacts use uniform RI context accumulation with 200 dimensions and eight nonzeros, followed by one residual graph average, E=(1-α)E0+αPE0, where P is a row-normalized PPMI graph and α=0.3. Terminal row normalization and per-dimension median/IQR scaling are then applied. On the Google analogy benchmark's family section, 272 of 506 questions are valid for every seed. Across five paired seeds, the complete pipeline raises accuracy from 19.41% to 30.74%, a gain of 11.32 percentage points with a nested-bootstrap 95% confidence interval of [6.93, 15.89]. Robust scaling alone contributes 3.24 points [1.25, 5.38], while graph averaging without robust scaling contributes 6.18 points [2.63, 9.92]. A separate 40-question general grid does not support a general improvement: the full pipeline changes accuracy by -6.00 points [-13.50, -0.50], and averaging without robust scaling changes it by -6.50 points [-14.50, -0.50]. The supported positive claim is therefore limited to the covered fairytales kinship analogy set; the results do not establish a generally effective embedding method.