mlsanityData to 7 Oct 2026

Unifying Data Perspectivism and Personalization: An Application to Social Norms

arXiv 2210.14531 ↗First posted 26 Oct 2022cs.CLcited by 22

Not on the map. Why?

Abstract (arXiv)

Instead of using a single ground truth for language processing tasks, several recent studies have examined how to represent and predict the labels of the set of annotators. However, often little or no information about annotators is known, or the set of annotators is small. In this work, we examine a corpus of social media posts about conflict from a set of 13k annotators and 210k judgements of social norms. We provide a novel experimental setup that applies personalization methods to the modeling of annotators and compare their effectiveness for predicting the perception of social norms. We further provide an analysis of performance across subsets of social situations that vary by the closeness of the relationship between parties in conflict, and assess where personalization helps the most.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 29 of its 60 references that are in the map. First posted on arXiv 26 Oct 2022.

Builds on

29 of 60 references.

References · 29

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  2. 2017Graph Attention Networkscited by 8,854
  3. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  4. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 951
  5. 2020GoEmotions: A Dataset of Fine-Grained Emotionscited by 479
  6. 2021Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotationscited by 306
  7. 2021Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detectioncited by 258
  8. 2020Social Chemistry 101: Learning to Reason about Social and Moral Normscited by 253
  9. 2019Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasetscited by 173
  10. 2021Toward a Perspectivist Turn in Ground Truthing for Predictive Computingcited by 129
  11. 2021On Releasing Annotator-Level Labels and Information in Datasetscited by 123
  12. 2020Scruples: A Corpus of Community Ethical Judgments on 32,000 Real-Life Anecdotescited by 100
17 more
  1. 2021Agreeing to Disagree: Annotating Offensive Language Datasets with Annotators' Disagreementcited by 71
  2. 2021UserIdentifier: Implicit User Representations for Simple and Effective Personalized Sentiment Analysiscited by 28
  3. 2018Personalized Language Model for Query Auto-Completioncited by 25
  4. 2021Conversational Question Answering over Knowledge Graphs with Transformer and Graph Attention Networkscited by 24
  5. 2015Overcoming Language Variation in Sentiment Analysis with Social Attentioncited by 23
  6. 2020PTUM: Pre-training User Model from Unlabeled User Behaviors via Self-supervisioncited by 22
  7. 2020Graph-based Modeling of Online Communities for Fake News Detectioncited by 21
  8. 2020Compositional Demographic Word Embeddingscited by 20
  9. 2020Exploring the Value of Personalized Word Embeddingscited by 17
  10. 2020Misinformation Has High Perplexitycited by 15
  11. 2022FACTOID: A New Dataset for Identifying Misinformation Spreaders and Political Biascited by 9
  12. 2018Representing Social Media Users for Sarcasm Detectioncited by 8
  13. 2021Analysis of Moral Judgement on Redditcited by 8
  14. 2019Exploring Author Context for Detecting Intended vs Perceived Sarcasmcited by 7
  15. 2022Explainable Patterns for Distinction and Prediction of Moral Judgement on Redditcited by 6
  16. 2022Understanding Interpersonal Conflict Types and their Impact on Perception Classificationcited by 4
  17. 2021Perceived and Intended Sarcasm Detection with Graph Attention Networkscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 10; 2024: 5; 2025: 4; 2026: 3.

22 papers.

Sort:

By area:

  1. 2023LaMP: When Large Language Models Meet Personalizationcited by 425
  2. 2023Pearl: Personalizing Large Language Model Writing Assistants with Generation-Calibrated Retrieverscited by 91
  3. 2023When the Majority is Wrong: Modeling Annotator Disagreement for Subjective Taskscited by 89
  4. 2024LLM-Rubric: A Multidimensional, Calibrated Approach to Automated Evaluation of Natural Language Textscited by 81
  5. 2023You Are What You Annotate: Towards Better Models through Annotator Representationscited by 40
  6. 2024The Perspectivist Paradigm Shift: Assumptions and Challenges of Capturing Human Labelscited by 37
  7. 2023Capturing Perspectives of Crowdsourced Annotators in Subjective Learning Taskscited by 28
  8. 2023Sociocultural Norm Similarities and Differences via Situational Alignment and Explainable Textual Entailmentcited by 22
  9. 2024Which Demographics do LLMs Default to During Annotation?cited by 10
  10. 2024Corpus Considerations for Annotator Modeling and Scalingcited by 8
  11. 2023Ensembled Prediction Intervals for Causal Outcomes Under Hidden Confoundingcited by 7
  12. 2023Your spouse needs professional help: Determining the Contextual Appropriateness of Messages through Modeling Social Relationshipscited by 6
10 more
  1. 2023Architectural Sweet Spots for Modeling Human Label Variation by the Example of Argument Quality: It's Best to Relate Perspectives!cited by 5
  2. 2024Estimating Causal Effects of Text Interventions Leveraging LLMscited by 4
  3. 2025VideoNorms: Benchmarking Cultural Awareness of Video Language Modelscited by 4
  4. 2025The Impact of Annotator Personas on LLM Behavior Across the Perspectivism Spectrumcited by 3
  5. 2023Personality-aware Human-centric Multimodal Reasoning: A New Task, Dataset and Baselinescited by 2
  6. 2025SOLAR: Towards Characterizing Subjectivity of Individuals through Modeling Value Conflicts and Trade-offscited by 0
  7. 2025Examining the Utility of Self-disclosure Types for Modeling Annotators of Social Normscited by 0
  8. 2026Not All Subjectivity Is the Same! Defining Desiderata for the Evaluation of Subjectivity in NLPcited by 0
  9. 2026Beyond Black-Box Labels: Interpretable Criteria for Diagnosing Subjective NLP Taskscited by 0
  10. 2026Fine-Grained Perspectives: Modeling Explanations with Annotator-Specific Rationalescited by 0