# Individual Text Corpora Predict Openness, Interests, Knowledge and Level of Education

arXiv: https://arxiv.org/abs/2404.00165 · HTML: https://mlsanity.com/paper/2404.00165/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 58 references that are in the map.
- First posted on arXiv 29 Mar 2024.

## Builds on (most cited first, 5 of 5)

- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [A high-bias, low-variance introduction to Machine Learning for physicists](https://mlsanity.com/paper/1803.08823/) (2018)
- [The Shape of Learning Curves: a Review](https://mlsanity.com/paper/2103.10948/) (2021)
- [Language Models Explain Word Reading Times Better Than Empirical Predictability](https://mlsanity.com/paper/2202.01128/) (2022)
- [Individual corpora predict fast memory retrieval during reading](https://mlsanity.com/paper/2010.10176/) (2020)

## Built on by (most cited first, 1 of 1)

- [From Retrieval to Weights: Parametric Individualization of Small Language Models with Individual Text Corpora](https://mlsanity.com/paper/2609.10155/) (2026)

## Abstract (arXiv)

Here we examine whether the personality dimension of openness to experience can be predicted from the individual google search history. By web scraping, individual text corpora (ICs) were generated from 214 participants with a mean number of 5 million word tokens. We trained word2vec models and used the similarities of each IC to label words, which were derived from a lexical approach of personality. These IC-label-word similarities were utilized as predictive features in neural models. For training and validation, we relied on 179 participants and held out a test sample of 35 participants. A grid search with varying number of predictive features, hidden units and boost factor was performed. As model selection criterion, we used R2 in the validation samples penalized by the absolute R2 difference between training and validation. The selected neural model explained 35% of the openness variance in the test sample, while an ensemble model with the same architecture often provided slightly more stable predictions for intellectual interests, knowledge in humanities and level of education. Finally, a learning curve analysis suggested that around 500 training participants are required for generalizable predictions. We discuss ICs as a complement or replacement of survey-based psychodiagnostics.