MoCA: Implicit Social Context Analysis
Not on the map. Why?
Abstract (arXiv)
Human social communication, such as affection and intent, is often conveyed in highly implicit ways, where underlying meanings are expressed through indirect, socially and culturally grounded signals rather than explicit statements. Such implicit social contexts are pervasive in real-world interactions, yet there remains a lack of a formal and systematic framework for studying them. In this paper, we introduce Implicit Social Context Analysis (MoCA), a novel task that systematically models implicit social scenarios along three key dimensions: affection, intent, and stance. We construct a high-quality benchmark containing 3,108 multimodal instances collected from real-world sources, with fine-grained cognitive annotations revealing who expresses what toward whom, as well as how and why it is conveyed. Using the MoCA dataset, we show that state-of-the-art multimodal large language models struggle significantly with this task because of their reliance on explicit cues and limited ability to reason over latent social contexts. To address this challenge, we propose Conflict-Driven Abductive Reasoning (CoDAR), a novel framework that models the discrepancy between observed expressions and expected truthful behavior as cognitive conflict, thereby enabling the inference of hidden mental states. Extensive experiments demonstrate that CoDAR substantially improves model performance. Nevertheless, a large gap from human reasoning remains, highlighting the fundamental difficulty of implicit social understanding.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 35 of its 77 references that are in the map. First posted on arXiv 6 Aug 2026.
Builds on
35 of 77 references.
References · 35
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2025Qwen3-VL Technical Reportcited by 2,793
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
- 2018MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversationscited by 643
- 2020The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memescited by 490
- 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learningcited by 466
- 2019COMET: Commonsense Transformers for Automatic Knowledge Graph Constructioncited by 450
- 2019Abductive Commonsense Reasoningcited by 301
- 2020Social Chemistry 101: Learning to Reason about Social and Moral Normscited by 255
- 2025MiniCPM-V 4.5: Cooking Efficient MLLMs via Architecture, Data, and Training Recipecited by 186
23 more
- 2021RedCaps: web-curated image-text data created by the people, for the peoplecited by 180
- 2019Towards Multimodal Sarcasm Detection (An _Obviously_ Perfect Paper)cited by 174
- 2024Video-of-Thought: Step-by-Step Video Reasoning from Perception to Cognitioncited by 153
- 2024Improve Vision Language Model Chain-of-thought Reasoningcited by 136
- 2024Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction Tuningcited by 121
- 2024MMToM-QA: Multimodal Theory of Mind Question Answeringcited by 85
- 2023Reasoning Implicit Sentiment with Chain-of-Thought Promptingcited by 80
- 2020Recent Neural Methods on Slot Filling and Intent Classification for Task-Oriented Dialogue Systems: A Surveycited by 51
- 2024MuMA-ToM: Multi-modal Multi-Agent Theory of Mindcited by 51
- 2023MemeCap: A Dataset for Captioning and Interpreting Memescited by 46
- 2022MIntRec: A New Dataset for Multimodal Intent Recognitioncited by 41
- 2023MMSD2.0: Towards a Reliable Multi-modal Sarcasm Detection Systemcited by 31
- 2024MIntRec2.0: A Large-scale Benchmark Dataset for Multimodal Intent Recognition and Out-of-scope Detection in Conversationscited by 26
- 2025Social Genome: Grounded Social Reasoning Abilities of Multimodal Modelscited by 26
- 2024A Survey of Multimodal Sarcasm Detectioncited by 19
- 2024Multi-modal Stance Detection: New Datasets and Modelcited by 18
- 2025MOMENTS: A Comprehensive Multimodal Benchmark for Theory of Mindcited by 13
- 2025Emotion-Qwen: A Unified Framework for Emotion and Vision Understandingcited by 11
- 2021A Survey of Hybrid Human-Artificial Intelligence for Social Computingcited by 5
- 2025MUStReason: A Benchmark for Diagnosing Pragmatic Reasoning in Video-LMs for Multimodal Sarcasm Detectioncited by 5
- 2023Affective Visual Dialog: A Large-Scale Benchmark for Emotional Reasoning Based on Visually Grounded Conversationscited by 4
- 2025VEGAS: Towards Visually Explainable and Grounded Artificial Social Intelligencecited by 4
- 2025DixitWorld: Evaluating Multimodal Abductive Reasoning in Vision-Language Models with Multi-Agent Dixit Gameplaycited by 4
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Towards Social AI: A Survey on Understanding Social Interactions
- 2026PIVOTSBench: Evaluating Fine-Grained Interpersonal Relationship Reasoning in Multimodal Large Language Models
- 2026GesBench: When and Where Do Multimodal Language Models Perceive Social Meaning?
- 2025SocialMaze: A Benchmark for Evaluating and Enhancing Social Reasoning in Large Language Models in Complex Social Environments
- 2025SIV-Bench: A Video Benchmark for Social Interaction Understanding and Reasoning
- 2026CEI: A Benchmark for Evaluating Pragmatic Reasoning in Language Models
- 2026COSI-Lab: Conference Living Lab for Modeling Multi-Perspective Multimodal Social Intention
- 2026Social Caption: Evaluating Social Understanding in Multimodal Models