# Towards Socially Intelligent Agents with Mental State Transition and Human Utility

arXiv: https://arxiv.org/abs/2103.07011 · HTML: https://mlsanity.com/paper/2103.07011/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 72 references that are in the map.
- First posted on arXiv 12 Mar 2021.

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation](https://mlsanity.com/paper/1911.00536/) (2019)
- [Wizard of Wikipedia: Knowledge-Powered Conversational agents](https://mlsanity.com/paper/1811.01241/) (2018)
- [TextWorld: A Learning Environment for Text-based Games](https://mlsanity.com/paper/1806.11532/) (2018)
- [Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding](https://mlsanity.com/paper/1810.02338/) (2018)
- [Bidirectional Attention Flow for Machine Comprehension](https://mlsanity.com/paper/1611.01603/) (2016)
- [Social Chemistry 101: Learning to Reason about Social and Moral Norms](https://mlsanity.com/paper/2011.00620/) (2020)
- [Neural-Symbolic Learning and Reasoning: A Survey and Interpretation](https://mlsanity.com/paper/1711.03902/) (2017)
- [QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension](https://mlsanity.com/paper/1804.09541/) (2018)
- [Learning to Speak and Act in a Fantasy Text Adventure Game](https://mlsanity.com/paper/1903.03094/) (2019)
- [Task-Oriented Dialogue as Dataflow Synthesis](https://mlsanity.com/paper/2009.11423/) (2020)
- [Language Understanding for Text-based Games Using Deep Reinforcement Learning](https://mlsanity.com/paper/1506.08941/) (2015)
- [SOLOIST: Building Task Bots at Scale with Transfer Learning and Machine Teaching](https://mlsanity.com/paper/2005.05298/) (2020)
- [Deep Reinforcement Learning for Mention-Ranking Coreference Models](https://mlsanity.com/paper/1609.08667/) (2016)
- [Learning Dynamic Belief Graphs to Generalize on Text-Based Games](https://mlsanity.com/paper/2002.09127/) (2020)
- [Counting to Explore and Generalize in Text-based Games](https://mlsanity.com/paper/1806.11525/) (2018)
- [How to Motivate Your Dragon: Teaching Goal-Driven Agents to Speak and Act in Fantasy Worlds](https://mlsanity.com/paper/2010.00685/) (2020)
- [Learning Executable Semantic Parsers for Natural Language Understanding](https://mlsanity.com/paper/1603.06677/) (2016)
- [Structured Attention for Unsupervised Dialogue Structure Induction](https://mlsanity.com/paper/2009.08552/) (2020)
- [LeDeepChef: Deep Reinforcement Learning Agent for Families of Text-Based Games](https://mlsanity.com/paper/1909.01646/) (2019)
- [Language (Re)modelling: Towards Embodied Language Understanding](https://mlsanity.com/paper/2005.00311/) (2020)
- [A Roadmap towards Machine Intelligence](https://mlsanity.com/paper/1511.08130/) (2015)
- [Comprehensible Context-driven Text Game Playing](https://mlsanity.com/paper/1905.02265/) (2019)
- [X-ToM: Explaining with Theory-of-Mind for Gaining Justified Human Trust](https://mlsanity.com/paper/1909.06907/) (2019)
- [Virtual Embodiment: A Scalable Long-Term Strategy for Artificial Intelligence Research](https://mlsanity.com/paper/1610.07432/) (2016)

## Built on by (most cited first, 14 of 14)

- [A Survey of Deep Learning for Mathematical Reasoning](https://mlsanity.com/paper/2212.10535/) (2022)
- [Minding Language Models' (Lack of) Theory of Mind: A Plug-and-Play Multi-Character Belief Tracker](https://mlsanity.com/paper/2306.00924/) (2023)
- [Towards A Holistic Landscape of Situated Theory of Mind in Large Language Models](https://mlsanity.com/paper/2310.19619/) (2023)
- [Do LLMs have Consistent Values?](https://mlsanity.com/paper/2407.12878/) (2024)
- [Limits of Theory of Mind Modelling in Dialogue-Based Collaborative Plan Acquisition](https://mlsanity.com/paper/2405.12621/) (2024)
- [SPRING: Situated Conversation Agent Pretrained with Multimodal Questions from Incremental Layout Graph](https://mlsanity.com/paper/2301.01949/) (2023)
- [GraphWOZ: Dialogue Management with Conversational Knowledge Graphs](https://mlsanity.com/paper/2211.12852/) (2022)
- [Towards a Neural Era in Dialogue Management for Collaboration: A Literature Survey](https://mlsanity.com/paper/2307.09021/) (2023)
- [Choice-75: A Dataset on Decision Branching in Script Learning](https://mlsanity.com/paper/2309.11737/) (2023)
- [Schema-Guided Culture-Aware Complex Event Simulation with Multi-Agent Role-Play](https://mlsanity.com/paper/2410.18935/) (2024)
- [Structuring the Space of Sociotechnical Alignment](https://mlsanity.com/paper/2607.01250/) (2026)
- [MindDial: Belief Dynamics Tracking with Theory-of-Mind Modeling for Situated Neural Dialogue Generation](https://mlsanity.com/paper/2306.15253/) (2023)
- [Human Emotion Verification by Action Languages via Answer Set Programming](https://mlsanity.com/paper/2601.12912/) (2026)
- [Graph Integrated Language Transformers for Next Action Prediction in Complex Phone Calls](https://mlsanity.com/paper/2404.08155/) (2024)

## Abstract (arXiv)

Building a socially intelligent agent involves many challenges. One of which is to track the agent's mental state transition and teach the agent to make decisions guided by its value like a human. Towards this end, we propose to incorporate mental state simulation and value modeling into dialogue agents. First, we build a hybrid mental state parser that extracts information from both the dialogue and event observations and maintains a graphical representation of the agent's mind; Meanwhile, the transformer-based value model learns human preferences from the human value dataset, ValueNet. Empirical results show that the proposed model attains state-of-the-art performance on the dialogue/action/emotion prediction task in the fantasy text-adventure game dataset, LIGHT. We also show example cases to demonstrate: (i) how the proposed mental state parser can assist the agent's decision by grounding on the context like locations and objects, and (ii) how the value model can help the agent make decisions based on its personal priorities.