# SI-Bench: Benchmarking Social Intelligence of Large Language Models in Human-to-Human Conversations

arXiv: https://arxiv.org/abs/2510.23182 · HTML: https://mlsanity.com/paper/2510.23182/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 38 references that are in the map.
- First posted on arXiv 27 Oct 2025.

## Builds on (most cited first, 6 of 6)

- [Social Chemistry 101: Learning to Reason about Social and Moral Norms](https://mlsanity.com/paper/2011.00620/) (2020)
- [Mind Your Step (by Step): Chain-of-Thought can Reduce Performance on Tasks where Thinking Makes Humans Worse](https://mlsanity.com/paper/2410.21333/) (2024)
- [Cue-CoT: Chain-of-thought Prompting for Responding to In-depth Dialogue Questions with LLMs](https://mlsanity.com/paper/2305.11792/) (2023)
- [Dialogue Chain-of-Thought Distillation for Commonsense-aware Conversational Agents](https://mlsanity.com/paper/2310.09343/) (2023)
- [AgentSense: Benchmarking Social Intelligence of Language Agents through Interactive Scenarios](https://mlsanity.com/paper/2410.19346/) (2024)
- [Academically intelligent LLMs are not necessarily socially intelligent](https://mlsanity.com/paper/2403.06591/) (2024)

## Built on by (most cited first, 7 of 7)

- [One Model, All Roles: Multi-Turn, Multi-Agent Self-Play Reinforcement Learning for Conversational Social Intelligence](https://mlsanity.com/paper/2602.03109/) (2026)
- [EIBench: A Simulator-Based Benchmark and Turn-Credit RL for Emotion Management](https://mlsanity.com/paper/2606.15532/) (2026)
- [Counterparty Modeling is Not Strategy: The Limits of LLM Negotiators](https://mlsanity.com/paper/2605.16575/) (2026)
- [AlloSpatial: Agentic Harness Framework for Spatial Reasoning in Foundation Models](https://mlsanity.com/paper/2606.08952/) (2026)
- [World2Mind: Cognition Toolkit for Allocentric Spatial Reasoning in Foundation Models](https://mlsanity.com/paper/2603.09774/) (2026)
- [ZenGen: Social Mind for LLMs](https://mlsanity.com/paper/2607.23740/) (2026)
- [Large-scale factor analysis shows machine intelligence is only partially interpretable](https://mlsanity.com/paper/2609.36515/) (2026)

## Abstract (arXiv)

As large language models (LLMs) develop anthropomorphic abilities, they are increasingly being deployed as autonomous agents to interact with humans. However, evaluating their performance in realistic and complex social interactions remains a significant challenge. Most previous research built datasets through simulated agent-to-agent interactions, which fails to capture the authentic linguistic styles and relational dynamics found in real human conversations. To address this gap, we introduce SI-Bench, a novel benchmark designed to evaluate aspects of social intelligence in LLMs. Grounded in broad social science theories, SI-Bench contains 2,221 authentic multi-turn dialogues collected from a social networking application. We further selected a subset of 312 dialogues for manual annotation across 8 major models. The experiments show that SOTA models have surpassed the human expert in process reasoning under complex social situations, yet they still fall behind humans in reply quality. Moreover, introducing Chain-of-Thought (CoT) reasoning may degrade the performance of LLMs in social dialogue tasks. All datasets are openly available at https://github.com/SI-Bench/SI-Bench.git.