mlsanityData to 7 Oct 2026

Large Human Language Models: A Need and the Challenges

arXiv 2312.07751 ↗First posted 9 Nov 2023cs.CL cs.AI cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

As research in human-centered NLP advances, there is a growing recognition of the importance of incorporating human and social factors into NLP models. At the same time, our NLP systems have become heavily reliant on LLMs, most of which do not model authors. To build NLP systems that can truly understand human language, we must better integrate human contexts into LLMs. This brings to the fore a range of design considerations and challenges in terms of what human aspects to capture, how to represent them, and what modeling strategies to pursue. To address these, we advocate for three positions toward creating large human language models (LHLMs) using concepts from psychological and behavioral sciences: First, LM training should include the human context. Second, LHLMs should recognize that people are more than their group(s). Third, LHLMs should be able to account for the dynamic and temporally-dependent nature of the human context. We refer to relevant advances and present open challenges that need to be addressed and their possible solutions in realizing these goals.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 47 of its 119 references that are in the map. First posted on arXiv 9 Nov 2023.

Builds on

47 of 119 references.

References · 47

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  5. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
  6. 2020Reformer: The Efficient Transformercited by 1,658
  7. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
  8. 2019Compressive Transformers for Long-Range Sequence Modellingcited by 699
  9. 2020Efficient Content-Based Sparse Attention with Routing Transformerscited by 469
  10. 2020Null It Out: Guarding Protected Attributes by Iterative Nullspace Projectioncited by 442
  11. 2023LaMP: When Large Language Models Meet Personalizationcited by 425
  12. 2023From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Modelscited by 257
35 more
  1. 2020Experience Grounds Languagecited by 241
  2. 2023Chat-REC: Towards Interactive and Explainable LLMs-Augmented Recommender Systemcited by 241
  3. 2021Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasetscited by 179
  4. 2020Towards Debiasing Sentence Representationscited by 170
  5. 2019Blockwise Self-Attention for Long Document Understandingcited by 158
  6. 2019Adaptive Attention Span in Transformerscited by 148
  7. 2022Probing Pre-Trained Language Models for Cross-Cultural Differences in Valuescited by 147
  8. 2019Predictive Biases in Natural Language Processing Models: A Conceptual Framework and Overviewcited by 137
  9. 2021Neural Machine Translation for Low-Resource Languages: A Surveycited by 108
  10. 2022The Ghost in the Machine has an American accent: value conflict in GPT-3cited by 105
  11. 2022Towards Leaving No Indic Language Behind: Building Monolingual Corpora, Benchmark and Models for Indic Languagescited by 99
  12. 2023AfriSenti: A Twitter Sentiment Analysis Benchmark for African Languagescited by 68
  13. 2023Do LLMs Understand Social Knowledge? Evaluating the Sociability of Large Language Models with SocKET Benchmarkcited by 62
  14. 2021Personalized Transformer for Explainable Recommendationcited by 58
  15. 2016Personalized Machine Translation: Preserving Original Author Traitscited by 47
  16. 2019BP-Transformer: Modelling Long-Range Context via Binary Partitioningcited by 38
  17. 2020Temporal Common Sense Acquisition with Minimal Supervisioncited by 38
  18. 2020Double-Hard Debias: Tailoring Word Embeddings for Gender Bias Mitigationcited by 36
  19. 2021UserIdentifier: Implicit User Representations for Simple and Effective Personalized Sentiment Analysiscited by 28
  20. 2016Modelling Context with User Embeddings for Sarcasm Detection in Social Mediacited by 26
  21. 2018Personalized Language Model for Query Auto-Completioncited by 25
  22. 2019Demographic Inference and Representative Population Estimates from Multilingual Social Media Datacited by 24
  23. 2015Overcoming Language Variation in Sentiment Analysis with Social Attentioncited by 23
  24. 2023Cross-Cultural Transfer Learning for Chinese Offensive Language Detectioncited by 21
  25. 2020Compositional Demographic Word Embeddingscited by 20
  26. 2019Learning to Answer by Learning to Ask: Getting the Best of GPT-2 and BERT Worldscited by 19
  27. 2020Exploring the Value of Personalized Word Embeddingscited by 17
  28. 2019You Shall Know a User by the Company It Keeps: Dynamic Representations for Social Media Users in NLPcited by 14
  29. 2021MeLT: Message-Level Transformer with Masked Document Representations as Pre-Training for Stance Detectioncited by 11
  30. 2017Latent Human Traits in the Language of Social Media: An Open-Vocabulary Approachcited by 7
  31. 2015Freshman or Fresher? Quantifying the Geographic Variation of Internet Languagecited by 6
  32. 2019Learning Dynamic Author Representations with Temporal Language Modelscited by 5
  33. 2020Author2Vec: A Framework for Generating User Embeddingcited by 5
  34. 2020Adding Recurrence to Pretrained Transformers for Improved Efficiency and Context Sizecited by 5
  35. 2015Learning multi-faceted representations of individuals from heterogeneous evidence using neural networkscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 4; 2025: 3; 2026: 3.

11 papers.

  1. 2024BIG5-CHAT: Shaping LLM Personalities Through Training on Human-Grounded Datacited by 49
  2. 2024The Potential and Challenges of Evaluating Attitudes, Opinions, and Values in Large Language Modelscited by 19
  3. 2024PLAYER*: Enhancing LLM-based Multi-Agent Communication and Interaction in Murder Mystery Gamescited by 10
  4. 2025From Evidence to Belief: A Bayesian Epistemology Approach to Language Modelscited by 5
  5. 2024Comparing Pre-trained Human Language Models: Is it Better with Human Context as Groups, Individual Traits, or Both?cited by 4
  6. 2023An HCAI Methodological Framework (HCAI-MF): Putting It Into Action to Enable Human-Centered AIcited by 3
  7. 2025WhiSPA: Semantically and Psychologically Aligned Whisper with Self-Supervised Contrastive and Student-Teacher Learningcited by 3
  8. 2025Evaluation of LLMs-based Hidden States as Author Representations for Psychological Human-Centered NLP Taskscited by 3
  9. 2026From Word Sequences to Behavioral Sequences: Adapting Modeling and Evaluation Paradigms for Longitudinal NLPcited by 3
  10. 2026Addressing the Ecological Fallacy in Larger LMs with Human Contextcited by 0
  11. 2026Who We Are, Where We Are: Mental Health at the Intersection of Person, Situation, and Large Language Modelscited by 0