# Learning Features from Co-occurrences: A Theoretical Analysis

arXiv: https://arxiv.org/abs/1707.04218 · HTML: https://mlsanity.com/paper/1707.04218/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Jul 2017.

## Built on by (most cited first, 1 of 1)

- [Nutribullets Hybrid: Multi-document Health Summarization](https://mlsanity.com/paper/2104.03465/) (2021)

## Abstract (arXiv)

Representing a word by its co-occurrences with other words in context is an effective way to capture the meaning of the word. However, the theory behind remains a challenge. In this work, taking the example of a word classification task, we give a theoretical analysis of the approaches that represent a word X by a function f(P(C|X)), where C is a context feature, P(C|X) is the conditional probability estimated from a text corpus, and the function f maps the co-occurrence measure to a prediction score. We investigate the impact of context feature C and the function f. We also explain the reasons why using the co-occurrences with multiple context features may be better than just using a single one. In addition, some of the results shed light on the theory of feature learning and machine learning in general.