# Storywrangler: A massive exploratorium for sociolinguistic, cultural, socioeconomic, and political timelines using Twitter

arXiv: https://arxiv.org/abs/2007.12988 · HTML: https://mlsanity.com/paper/2007.12988/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Jul 2020.

## Built on by (most cited first, 15 of 15)

- [Evolving linguistic divergence on polarizing social media](https://mlsanity.com/paper/2309.01659/) (2023)
- [Entropy and type-token ratio in gigaword corpora](https://mlsanity.com/paper/2411.10227/) (2024)
- [The incel lexicon: Deciphering the emergent cryptolect of a global misogynistic community](https://mlsanity.com/paper/2105.12006/) (2021)
- [American cultural regions mapped through the lexical analysis of social media](https://mlsanity.com/paper/2208.07649/) (2022)
- [Tweet Insights: A Visualization Platform to Extract Temporal Insights from Twitter](https://mlsanity.com/paper/2308.02142/) (2023)
- [Quantifying language changes surrounding mental health on Twitter](https://mlsanity.com/paper/2106.01481/) (2021)
- [Sentiment and structure in word co-occurrence networks on Twitter](https://mlsanity.com/paper/2110.00587/) (2021)
- [When Dialects Collide: How Socioeconomic Mixing Affects Language Use](https://mlsanity.com/paper/2307.10016/) (2023)
- [Augmenting semantic lexicons using word embeddings and transfer learning](https://mlsanity.com/paper/2109.09010/) (2021)
- [Computational Paremiology: Charting the temporal, ecological dynamics of proverb use in books, news articles, and tweets](https://mlsanity.com/paper/2107.04929/) (2021)
- [Ousiometrics: The essence of meaning aligns with a power-danger-structure framework instead of valence-arousal-dominance](https://mlsanity.com/paper/2110.06847/) (2021)
- [Language statistics at different spatial, temporal, and grammatical scales](https://mlsanity.com/paper/2207.00709/) (2022)
- [Curating corpora with classifiers: A case study of clean energy sentiment online](https://mlsanity.com/paper/2305.03092/) (2023)
- [An assessment of measuring local levels of homelessness through proxy social media signals](https://mlsanity.com/paper/2305.08978/) (2023)
- [When or What? Understanding Consumer Engagement on Digital Platforms](https://mlsanity.com/paper/2510.10474/) (2025)

## Abstract (arXiv)

In real-time, social media data strongly imprints world events, popular culture, and day-to-day conversations by millions of ordinary people at a scale that is scarcely conventionalized and recorded. Vitally, and absent from many standard corpora such as books and news archives, sharing and commenting mechanisms are native to social media platforms, enabling us to quantify social amplification (i.e., popularity) of trending storylines and contemporary cultural phenomena. Here, we describe Storywrangler, a natural language processing instrument designed to carry out an ongoing, day-scale curation of over 100 billion tweets containing roughly 1 trillion 1-grams from 2008 to 2021. For each day, we break tweets into unigrams, bigrams, and trigrams spanning over 100 languages. We track n-gram usage frequencies, and generate Zipf distributions, for words, hashtags, handles, numerals, symbols, and emojis. We make the data set available through an interactive time series viewer, and as downloadable time series and daily distributions. Although Storywrangler leverages Twitter data, our method of extracting and tracking dynamic changes of n-grams can be extended to any similar social media platform. We showcase a few examples of the many possible avenues of study we aim to enable including how social amplification can be visualized through 'contagiograms'. We also present some example case studies that bridge n-gram time series with disparate data sources to explore sociotechnical dynamics of famous individuals, box office success, and social unrest.