# SocialDial: A Benchmark for Socially-Aware Dialogue Systems

arXiv: https://arxiv.org/abs/2304.12026 · HTML: https://mlsanity.com/paper/2304.12026/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 51 references that are in the map.
- First posted on arXiv 24 Apr 2023.

## Builds on (most cited first, 20 of 20)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Personalizing Dialogue Agents: I have a dog, do you have pets too?](https://mlsanity.com/paper/1801.07243/) (2018)
- [Aligning AI With Shared Human Values](https://mlsanity.com/paper/2008.02275/) (2020)
- [Social Bias Frames: Reasoning about Social and Power Implications of Language](https://mlsanity.com/paper/1911.03891/) (2019)
- [Towards Emotional Support Dialog Systems](https://mlsanity.com/paper/2106.01144/) (2021)
- [Social Chemistry 101: Learning to Reason about Social and Moral Norms](https://mlsanity.com/paper/2011.00620/) (2020)
- [Persuasion for Good: Towards a Personalized Persuasive Dialogue System for Social Good](https://mlsanity.com/paper/1906.06725/) (2019)
- [Scruples: A Corpus of Community Ethical Judgments on 32,000 Real-Life Anecdotes](https://mlsanity.com/paper/2008.09094/) (2020)
- [The Moral Integrity Corpus: A Benchmark for Ethical Dialogue Systems](https://mlsanity.com/paper/2204.03021/) (2022)
- [Event2Mind: Commonsense Inference on Events, Intents, and Reactions](https://mlsanity.com/paper/1805.06939/) (2018)
- [PLACES: Prompting Language Models for Social Conversation Synthesis](https://mlsanity.com/paper/2302.03269/) (2023)
- [NormSAGE: Multi-Lingual Multi-Cultural Norm Discovery from Conversations On-the-Fly](https://mlsanity.com/paper/2210.08604/) (2022)
- [Towards an Atlas of Cultural Commonsense for Machine Reasoning](https://mlsanity.com/paper/2009.05664/) (2020)
- [CPED: A Large-Scale Chinese Personalized and Emotional Dialogue Dataset for Conversational AI](https://mlsanity.com/paper/2205.14727/) (2022)
- [DREAM: Improving Situational QA by First Elaborating the Situation](https://mlsanity.com/paper/2112.08656/) (2021)
- [Opponent Modeling in Negotiation Dialogues by Related Data Adaptation](https://mlsanity.com/paper/2205.00344/) (2022)
- [Keeping Up Appearances: Computational Modeling of Face Acts in Persuasion Oriented Discussions](https://mlsanity.com/paper/2009.10815/) (2020)
- [Towards Emotion-Aware Agents For Negotiation Dialogues](https://mlsanity.com/paper/2107.13165/) (2021)

## Built on by (most cited first, 22 of 22)

- [Towards Measuring and Modeling "Culture" in LLMs: A Survey](https://mlsanity.com/paper/2403.15412/) (2024)
- [Natural Language Processing for Dialects of a Language: A Survey](https://mlsanity.com/paper/2401.05632/) (2024)
- [Culturally Aware and Adapted NLP: A Taxonomy and a Survey of the State of the Art](https://mlsanity.com/paper/2406.03930/) (2024)
- [SocialMind: LLM-based Proactive AR Social Assistive System with Human-like Perception for In-situ Live Interactions](https://mlsanity.com/paper/2412.04036/) (2024)
- [Leveraging Large Language Models for Automated Dialogue Analysis](https://mlsanity.com/paper/2309.06490/) (2023)
- [RENOVI: A Benchmark Towards Remediating Norm Violations in Socio-Cultural Conversations](https://mlsanity.com/paper/2402.11178/) (2024)
- [Conversational Feedback in Scripted versus Spontaneous Dialogues: A Comparative Analysis](https://mlsanity.com/paper/2309.15656/) (2023)
- [NormMark: A Weakly Supervised Markov Model for Socio-cultural Norm Discovery](https://mlsanity.com/paper/2305.16598/) (2023)
- [Developing a Tutoring Dialog Dataset to Optimize LLMs for Educational Use](https://mlsanity.com/paper/2410.19231/) (2024)
- [ConvoSense: Overcoming Monotonous Commonsense Inferences for Conversational AI](https://mlsanity.com/paper/2401.15471/) (2024)
- [RoleConflictBench: A Benchmark of Role Conflict Scenarios for Evaluating LLMs' Contextual Sensitivity](https://mlsanity.com/paper/2509.25897/) (2025)
- [Are they lovers or friends? Evaluating LLMs' Social Reasoning in English and Korean Dialogues](https://mlsanity.com/paper/2510.19028/) (2025)
- [SADAS: A Dialogue Assistant System Towards Remediating Norm Violations in Bilingual Socio-Cultural Conversations](https://mlsanity.com/paper/2402.01736/) (2024)
- [KoDialogBench: Evaluating Conversational Understanding of Language Models with Korean Dialogue Benchmark](https://mlsanity.com/paper/2402.17377/) (2024)
- [EgoSocial: Benchmarking Proactive Intervention Ability of Omnimodal LLMs via Egocentric Social Interaction Perception](https://mlsanity.com/paper/2510.13105/) (2025)
- [Scalable Frame-based Construction of Sociocultural NormBases for Socially-Aware Dialogues](https://mlsanity.com/paper/2410.03049/) (2024)
- [EUDAIMONIA: Evaluating Undesirable Dynamics in AI](https://mlsanity.com/paper/2605.30654/) (2026)
- [Let's Negotiate! A Survey of Negotiation Dialogue Systems](https://mlsanity.com/paper/2402.01097/) (2024)
- [RoTRAG: Rule of Thumb Reasoning for Conversation Harm Detection with Retrieval-Augmented Generation](https://mlsanity.com/paper/2604.17301/) (2026)
- [TIDES: A Longitudinal Bilingual Dataset for Modeling Multi-Party Social Dynamics](https://mlsanity.com/paper/2608.01724/) (2026)
- [Beyond Right and Wrong: Evaluating Second-order Social Reasoning in Large Language Models](https://mlsanity.com/paper/2609.05437/) (2026)
- [FineWeb-CLaR: Culture, Language, and Region Annotations for Benchmark-Aligned Corpus Auditing](https://mlsanity.com/paper/2609.25298/) (2026)

## Abstract (arXiv)

Dialogue systems have been widely applied in many scenarios and are now more powerful and ubiquitous than ever before. With large neural models and massive available data, current dialogue systems have access to more knowledge than any people in their life. However, current dialogue systems still do not perform at a human level. One major gap between conversational agents and humans lies in their abilities to be aware of social norms. The development of socially-aware dialogue systems is impeded due to the lack of resources. In this paper, we present the first socially-aware dialogue corpus - SocialDial, based on Chinese social culture. SocialDial consists of two parts: 1,563 multi-turn dialogues between two human speakers with fine-grained labels, and 4,870 synthetic conversations generated by ChatGPT. The human corpus covers five categories of social norms, which have 14 sub-categories in total. Specifically, it contains social factor annotations including social relation, context, social distance, and social norms. However, collecting sufficient socially-aware dialogues is costly. Thus, we harness the power of ChatGPT and devise an ontology-based synthetic data generation framework. This framework is able to generate synthetic data at scale. To ensure the quality of synthetic dialogues, we design several mechanisms for quality control during data collection. Finally, we evaluate our dataset using several pre-trained models, such as BERT and RoBERTa. Comprehensive empirical results based on state-of-the-art neural models demonstrate that modeling of social norms for dialogue systems is a promising research direction. To the best of our knowledge, SocialDial is the first socially-aware dialogue dataset that covers multiple social factors and has fine-grained labels.