Situated, Dynamic, and Subjective: Envisioning the Design of Theory-of-Mind-Enabled Everyday AI with Industry Practitioners
Not on the map. Why?
Abstract (arXiv)
Theory of Mind (ToM) -- the ability to infer what others are thinking (e.g., intentions) from observable cues -- is traditionally considered fundamental to human social interactions. This has sparked growing efforts in building and benchmarking AI's ToM capability, yet little is known about how such capability could translate into the design and experience of everyday user-facing AI products and services. We conducted 13 co-design sessions with 26 U.S.-based AI practitioners to envision, reflect, and distill design recommendations for ToM-enabled everyday AI products and services that are both future-looking and grounded in the realities of AI design and development practices. Analysis revealed three interrelated design recommendations: ToM-enabled AI should 1) be situated in the social context that shape users' mental states, 2) be responsive to the dynamic nature of mental states, and 3) be attuned to subjective individual differences. We surface design tensions within each recommendation that reveal a broader gap between practitioners' envisioned futures of ToM-enabled AI and the realities of current AI design and development practices. These findings point toward the need to move beyond static, inference-driven approach to ToM and toward designing ToM as a pervasive capability that supports continuous human-AI interaction loops.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 17 of its 93 references that are in the map. First posted on arXiv 11 Feb 2026.
Builds on
17 of 93 references.
References · 17
- 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
- 2023Large Language Models Fail on Trivial Alterations to Theory-of-Mind Taskscited by 288
- 2020When Humans Aren't Optimal: Robots that Collaborate with Risk-Aware Humanscited by 34
- 2023Towards A Holistic Landscape of Situated Theory of Mind in Large Language Modelscited by 32
- 2025On Benchmarking Human-Like Intelligence in Machinescited by 24
- 2025Re-evaluating Theory of Mind evaluation in large language modelscited by 23
- 2021CX-ToM: Counterfactual Explanations with Theory-of-Mind for Enhancing Human Trust in Image Recognition Modelscited by 20
- 2023Designerly Understanding: Information Needs for Model Transparency to Support Design Ideation for AI-Powered User Experiencecited by 18
- 2024Prototyping with Prompts: Emerging Approaches and Challenges in Generative AI Design for Collaborative Software Teamscited by 16
- 2022Mutual Theory of Mind for Human-AI Communicationcited by 15
- 2025Rethinking Theory of Mind Benchmarks for LLMs: Towards A User-Centered Perspectivecited by 14
- 2023fAIlureNotes: Supporting Designers in Understanding the Limits of AI Models for Computer Vision Taskscited by 10
5 more
- 2021Eliciting and Analysing Users' Envisioned Dialogues with Perfect Voice Assistantscited by 9
- 2023A Review on Machine Theory of Mindcited by 9
- 2024Sketching AI Concepts with Capabilities and Examples: AI Innovation in the Intensive Care Unitcited by 6
- 2025"What If Smart Homes Could See Our Homes?": Exploring DIY Smart Home Building Experiences with VLM-Based Camera Sensorscited by 3
- 2025Unremarkable to Remarkable AI Agent: Exploring Boundaries of Agent Intervention for Adults With and Without Cognitive Impairmentcited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.