mlsanityData to 14 Oct 2026

Social Perceptions of English Spelling Variation on Twitter: A Comparative Analysis of Human and LLM Responses

arXiv 2511.23041 ↗First posted 28 Nov 2025cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Spelling variation (e.g. funnnn vs. fun) can influence the social perception of texts and their writers: we often have various associations with different forms of writing (is the text informal? does the writer seem young?). In this study, we focus on the social perception of spelling variation in online writing in English and study to what extent this perception is aligned between humans and large language models (LLMs). Building on sociolinguistic methodology, we compare LLM and human ratings on three key social attributes of spelling variation (formality, carefulness, age). We find generally strong correlations in the ratings between humans and LLMs. However, notable differences emerge when we analyze the distribution of ratings and when comparing between different types of spelling variation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 37 of its 120 references that are in the map. First posted on arXiv 28 Nov 2025.

Builds on

37 of 120 references.

References · 37

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  3. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 956
  4. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 817
  5. 2023Whose Opinions Do Language Models Reflect?cited by 773
  6. 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 752
  7. 2019BLiMP: The Benchmark of Linguistic Minimal Pairs for Englishcited by 564
  8. 2022Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studiescited by 515
  9. 2024Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Researchcited by 483
  10. 2023Towards Measuring the Representation of Subjective Global Opinions in Language Modelscited by 437
  11. 2023Language Model Tokenizers Introduce Unfairness Between Languagescited by 214
  12. 2018Dear Sir or Madam, May I introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfercited by 211
25 more
  1. 2023Do LLMs exhibit human-like response biases? A case study in survey designcited by 180
  2. 2020Deep Learning for Text Style Transfer: A Surveycited by 162
  3. 2024Large language models that replace human participants can harmfully misportray and flatten identity groupscited by 144
  4. 2025LLM Social Simulations Are a Promising Research Methodcited by 99
  5. 2023Sensitivity, Performance, Robustness: Deconstructing the Effect of Sociodemographic Promptingcited by 81
  6. 2024Linguistic Bias in ChatGPT: Language Models Reinforce Dialect Discriminationcited by 56
  7. 2022Multi-VALUE: A Framework for Cross-Dialectal English NLPcited by 53
  8. 2024DIALECTBENCH: A NLP Benchmark for Dialects, Varieties, and Closely-Related Languagescited by 50
  9. 2023You don't need a personality test to know these models are unreliable: Assessing the Reliability of Large Language Models on Psychometric Instrumentscited by 48
  10. 2025The Prompt Makes the Person(a): A Systematic Evaluation of Sociodemographic Persona Prompting for Large Language Modelscited by 42
  11. 2024Natural Language Processing for Dialects of a Language: A Surveycited by 40
  12. 2023Evaluation of African American Language Bias in Natural Language Generationcited by 36
  13. 2015Computational Sociolinguistics: A Surveycited by 35
  14. 2025The Shrinking Landscape of Linguistic Diversity in the Age of Large Language Modelscited by 29
  15. 2024Holmes: A Benchmark to Assess the Linguistic Competence of Language Modelscited by 26
  16. 2024Assessing Dialect Fairness and Robustness of Large Language Models in Reasoning Taskscited by 22
  17. 2025HumT DumT: Measuring and controlling human-like language in LLMscited by 12
  18. 2023"One-Size-Fits-All"? Examining Expectations around What Constitute "Fair" or "Good" NLG System Behaviorscited by 11
  19. 2019Adapting Sequence to Sequence models for Text Normalization in Social Mediacited by 8
  20. 2025Minimal Pair-Based Evaluation of Code-Switchingcited by 8
  21. 2022SocioProbe: What, When, and Where Language Models Learn about Sociodemographicscited by 7
  22. 2025Psycholinguistic Word Features: a New Approach for the Evaluation of LLMs Alignment with Humanscited by 5
  23. 2024Using large language models to estimate features of multi-word expressions: Concreteness, valence, arousalcited by 4
  24. 2023In What Languages are Generative Language Models the Most Formal? Analyzing Formality Distribution across Languagescited by 2
  25. 2025My LLM might Mimic AAE -- But When Should it?cited by 1

Built on by

Not cited yet.