# Social Intelligence Data Infrastructure: Structuring the Present and Navigating the Future

arXiv: https://arxiv.org/abs/2403.14659 · HTML: https://mlsanity.com/paper/2403.14659/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 122 references that are in the map.
- First posted on arXiv 28 Feb 2024.

## Builds on (most cited first, 30 of 42)

- [Personalizing Dialogue Agents: I have a dog, do you have pets too?](https://mlsanity.com/paper/1801.07243/) (2018)
- [MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling](https://mlsanity.com/paper/1810.00278/) (2018)
- [DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset](https://mlsanity.com/paper/1710.03957/) (2017)
- [GoEmotions: A Dataset of Fine-Grained Emotions](https://mlsanity.com/paper/2005.00547/) (2020)
- [Towards Measuring the Representation of Subjective Global Opinions in Language Models](https://mlsanity.com/paper/2306.16388/) (2023)
- [TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classification](https://mlsanity.com/paper/2010.12421/) (2020)
- [Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance](https://mlsanity.com/paper/2006.14779/) (2020)
- [Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces](https://mlsanity.com/paper/1805.10190/) (2018)
- [SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents](https://mlsanity.com/paper/2310.11667/) (2023)
- [Social Chemistry 101: Learning to Reason about Social and Moral Norms](https://mlsanity.com/paper/2011.00620/) (2020)
- [Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMs](https://mlsanity.com/paper/2210.13312/) (2022)
- [Persuasion for Good: Towards a Personalized Persuasive Dialogue System for Social Good](https://mlsanity.com/paper/1906.06725/) (2019)
- [Challenges in Building Intelligent Open-domain Dialog Systems](https://mlsanity.com/paper/1905.05709/) (2019)
- [A Benchmark Dataset for Learning to Intervene in Online Hate Speech](https://mlsanity.com/paper/1909.04251/) (2019)
- [CoMPosT: Characterizing and Evaluating Caricature in LLM Simulations](https://mlsanity.com/paper/2310.11501/) (2023)
- [CoAnnotating: Uncertainty-Guided Work Allocation between Human and Large Language Models for Data Annotation](https://mlsanity.com/paper/2310.15638/) (2023)
- [When to Make Exceptions: Exploring Language Models as Accounts of Human Moral Judgment](https://mlsanity.com/paper/2210.01478/) (2022)
- [Recent Advances and Challenges in Task-oriented Dialog System](https://mlsanity.com/paper/2003.07490/) (2020)
- [Corpus Creation for Sentiment Analysis in Code-Mixed Tamil-English Text](https://mlsanity.com/paper/2006.00206/) (2020)
- [CrossWOZ: A Large-Scale Chinese Cross-Domain Task-Oriented Dialogue Dataset](https://mlsanity.com/paper/2002.11893/) (2020)
- [Do LLMs Understand Social Knowledge? Evaluating the Sociability of Large Language Models with SocKET Benchmark](https://mlsanity.com/paper/2305.14938/) (2023)
- [CaSiNo: A Corpus of Campsite Negotiation Dialogues for Automatic Negotiation Systems](https://mlsanity.com/paper/2103.15721/) (2021)
- [MindCraft: Theory of Mind Modeling for Situated Dialogue in Collaborative Tasks](https://mlsanity.com/paper/2109.06275/) (2021)
- [NormBank: A Knowledge Bank of Situational Social Norms](https://mlsanity.com/paper/2305.17008/) (2023)
- [Will-They-Won't-They: A Very Large Dataset for Stance Detection on Twitter](https://mlsanity.com/paper/2005.00388/) (2020)
- [A Sentiment Analysis Dataset for Code-Mixed Malayalam-English](https://mlsanity.com/paper/2006.00210/) (2020)
- [Designing for Responsible Trust in AI Systems: A Communication Perspective](https://mlsanity.com/paper/2204.13828/) (2022)
- [iSarcasm: A Dataset of Intended Sarcasm](https://mlsanity.com/paper/1911.03123/) (2019)
- [CICERO: A Dataset for Contextualized Commonsense Inference in Dialogues](https://mlsanity.com/paper/2203.13926/) (2022)
- [RADDLE: An Evaluation Benchmark and Analysis Platform for Robust Task-oriented Dialog Systems](https://mlsanity.com/paper/2012.14666/) (2020)

## Built on by (most cited first, 11 of 11)

- [A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systems](https://mlsanity.com/paper/2504.09037/) (2025)
- [InterIntent: Investigating Social Intelligence of LLMs via Intention Understanding in an Interactive Game Context](https://mlsanity.com/paper/2406.12203/) (2024)
- [A Survey on Large Language Model-Based Social Agents in Game-Theoretic Scenarios](https://mlsanity.com/paper/2412.03920/) (2024)
- [Dialogue Action Tokens: Steering Language Models in Goal-Directed Dialogue with a Multi-Turn Planner](https://mlsanity.com/paper/2406.11978/) (2024)
- [EgoSocialArena: Benchmarking the Social Intelligence of Large Language Models from a First-person Perspective](https://mlsanity.com/paper/2410.06195/) (2024)
- [LIFELONG SOTOPIA: Evaluating Social Intelligence of Language Agents Over Lifelong Social Interactions](https://mlsanity.com/paper/2506.12666/) (2025)
- [HumanLLM: Towards Personalized Understanding and Simulation of Human Nature](https://mlsanity.com/paper/2601.15793/) (2026)
- [Fostering Natural Conversation in Large Language Models with NICO: a Natural Interactive COnversation dataset](https://mlsanity.com/paper/2408.09330/) (2024)
- [SAVOIR: Learning Social Savoir-Faire via Shapley-based Reward Attribution](https://mlsanity.com/paper/2604.18982/) (2026)
- [BenCSSmark: Making the Social Sciences Count in LLM Research](https://mlsanity.com/paper/2605.04886/) (2026)
- [PerSo: Investigating Facets of LLMs' Social Intelligence in Real-World Persuasion Scenarios](https://mlsanity.com/paper/2610.20548/) (2026)

## Abstract (arXiv)

As Natural Language Processing (NLP) systems become increasingly integrated into human social life, these technologies will need to increasingly rely on social intelligence. Although there are many valuable datasets that benchmark isolated dimensions of social intelligence, there does not yet exist any body of work to join these threads into a cohesive subfield in which researchers can quickly identify research gaps and future directions. Towards this goal, we build a Social AI Data Infrastructure, which consists of a comprehensive social AI taxonomy and a data library of 480 NLP datasets. Our infrastructure allows us to analyze existing dataset efforts, and also evaluate language models' performance in different social intelligence aspects. Our analyses demonstrate its utility in enabling a thorough understanding of current data landscape and providing a holistic perspective on potential directions for future dataset development. We show there is a need for multifaceted datasets, increased diversity in language and culture, more long-tailed social situations, and more interactive data in future social intelligence data efforts.