# Splits! Flexible Sociocultural Linguistic Investigation at Scale

arXiv: https://arxiv.org/abs/2504.04640 · HTML: https://mlsanity.com/paper/2504.04640/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 86 references that are in the map.
- First posted on arXiv 6 Apr 2025.

## Builds on (most cited first, 23 of 23)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT](https://mlsanity.com/paper/2004.12832/) (2020)
- [Language (Technology) is Power: A Critical Survey of "Bias" in NLP](https://mlsanity.com/paper/2005.14050/) (2020)
- [StereoSet: Measuring stereotypical bias in pretrained language models](https://mlsanity.com/paper/2004.09456/) (2020)
- [Whose Opinions Do Language Models Reflect?](https://mlsanity.com/paper/2303.17548/) (2023)
- [Social Bias Frames: Reasoning about Social and Power Implications of Language](https://mlsanity.com/paper/1911.03891/) (2019)
- [Large Language Models for Automated Open-domain Scientific Hypotheses Discovery](https://mlsanity.com/paper/2309.02726/) (2023)
- [ConvoKit: A Toolkit for the Analysis of Conversations](https://mlsanity.com/paper/2005.04246/) (2020)
- [Beyond prompt brittleness: Evaluating the reliability and consistency of political worldviews in LLMs](https://mlsanity.com/paper/2402.17649/) (2024)
- [VALUE: Understanding Dialect Disparity in NLU](https://mlsanity.com/paper/2204.03031/) (2022)
- [Interpretation of NLP models through input marginalization](https://mlsanity.com/paper/2010.13984/) (2020)
- [Natural Language Processing for Dialects of a Language: A Survey](https://mlsanity.com/paper/2401.05632/) (2024)
- [The Perspectivist Paradigm Shift: Assumptions and Challenges of Capturing Human Labels](https://mlsanity.com/paper/2405.05860/) (2024)
- [Does BERT Learn as Humans Perceive? Understanding Linguistic Styles through Lexica](https://mlsanity.com/paper/2109.02738/) (2021)
- [Interactive Concept Learning for Uncovering Latent Themes in Large Text Collections](https://mlsanity.com/paper/2305.05094/) (2023)
- [Building Knowledge-Guided Lexica to Model Cultural Variation](https://mlsanity.com/paper/2406.11622/) (2024)
- [Studying Politeness across Cultures Using English Twitter and Mandarin Weibo](https://mlsanity.com/paper/2008.02449/) (2020)
- [Investigating Human Values in Online Communities](https://mlsanity.com/paper/2402.14177/) (2024)
- [Using Noisy Self-Reports to Predict Twitter User Demographics](https://mlsanity.com/paper/2005.00635/) (2020)
- ["A Tale of Two Movements": Identifying and Comparing Perspectives in #BlackLivesMatter and #BlueLivesMatter Movements-related Tweets using Weakly Supervised Graph-based Structured Prediction](https://mlsanity.com/paper/2310.07155/) (2023)
- ["We Demand Justice!": Towards Social Context Grounding of Political Texts](https://mlsanity.com/paper/2311.09106/) (2023)
- [Subspace Representations for Soft Set Operations and Sentence Similarities](https://mlsanity.com/paper/2210.13034/) (2022)

## Abstract (arXiv)

Variation in language use, shaped by speakers' sociocultural background and specific context of use, offers a rich lens into cultural perspectives, values, and opinions. For example, Chinese students discuss "healthy eating" with words like "timing," "regularity," and "digestion," whereas Americans use vocabulary like "balancing food groups" and "avoiding fat and sugar," reflecting distinct cultural models of nutrition. The computational study of these Sociocultural Linguistic Phenomena (SLP) has traditionally been done in NLP via tailored analyses of specific groups or topics, requiring specialized data collection and experimental operationalization--a process not well-suited to quick hypothesis exploration and prototyping. To address this, we propose constructing a "sandbox" designed for systematic and flexible sociolinguistic research. Using our method, we construct a demographically/topically split Reddit dataset, Splits!, validated by self-identification and by replicating several known SLPs from existing literature. We showcase the sandbox's utility with a scalable, two-stage process that filters large collections of "potential" SLPs (PSLPs) to surface the most promising candidates for deeper, qualitative investigation.