# The Remarkable Benefit of User-Level Aggregation for Lexical-based Population-Level Predictions

arXiv: https://arxiv.org/abs/1808.09600 · HTML: https://mlsanity.com/paper/1808.09600/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Aug 2018.

## Built on by (most cited first, 10 of 10)

- [Building Knowledge-Guided Lexica to Model Cultural Variation](https://mlsanity.com/paper/2406.11622/) (2024)
- [Human Language Modeling](https://mlsanity.com/paper/2205.05128/) (2022)
- [Robust language-based mental health assessments in time and space through social media](https://mlsanity.com/paper/2302.12952/) (2023)
- [PsychAdapter: Adapting LLM Transformers to Reflect Traits, Personality and Mental Health](https://mlsanity.com/paper/2412.16882/) (2024)
- [LocalTweets to LocalHealth: A Mental Health Surveillance Framework Based on Twitter Data](https://mlsanity.com/paper/2402.13452/) (2024)
- [Impoverished Language Technology: The Lack of (Social) Class in NLP](https://mlsanity.com/paper/2403.03874/) (2024)
- [Improved Topic modeling in Twitter through Community Pooling](https://mlsanity.com/paper/2201.00690/) (2021)
- [Historical patterns of rice farming explain modern-day language use in China and Japan more than modernization and urbanization](https://mlsanity.com/paper/2308.15352/) (2023)
- [Inferring Effects of Major Events through Discontinuity Forecasting of Population Anxiety](https://mlsanity.com/paper/2508.21722/) (2025)
- [Word-Aligned Prosody Reflects Speaker Differences More Than Depression or PTSD Severity](https://mlsanity.com/paper/2610.15181/) (2026)

## Abstract (arXiv)

Nowcasting based on social media text promises to provide unobtrusive and near real-time predictions of community-level outcomes. These outcomes are typically regarding people, but the data is often aggregated without regard to users in the Twitter populations of each community. This paper describes a simple yet effective method for building community-level models using Twitter language aggregated by user. Results on four different U.S. county-level tasks, spanning demographic, health, and psychological outcomes show large and consistent improvements in prediction accuracies (e.g. from Pearson r=.73 to .82 for median income prediction or r=.37 to .47 for life satisfaction prediction) over the standard approach of aggregating all tweets. We make our aggregated and anonymized community-level data, derived from 37 billion tweets -- over 1 billion of which were mapped to counties, available for research.