# Baby Intuitions Benchmark (BIB): Discerning the goals, preferences, and actions of others

arXiv: https://arxiv.org/abs/2102.11938 · HTML: https://mlsanity.com/paper/2102.11938/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 83 references that are in the map.
- First posted on arXiv 23 Feb 2021.

## Builds on (most cited first, 13 of 13)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems](https://mlsanity.com/paper/2005.01643/) (2020)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [AI2-THOR: An Interactive 3D Environment for Visual AI](https://mlsanity.com/paper/1712.05474/) (2017)
- [Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variables](https://mlsanity.com/paper/1903.08254/) (2019)
- [Maximum a Posteriori Policy Optimisation](https://mlsanity.com/paper/1806.06920/) (2018)
- [Decoupling Representation Learning from Reinforcement Learning](https://mlsanity.com/paper/2009.08319/) (2020)
- [Machine Theory of Mind](https://mlsanity.com/paper/1802.07740/) (2018)
- [Keep Doing What Worked: Behavioral Modelling Priors for Offline Reinforcement Learning](https://mlsanity.com/paper/2002.08396/) (2020)
- [Autonomous Agents Modelling Other Agents: A Comprehensive Survey and Open Problems](https://mlsanity.com/paper/1709.08071/) (2017)
- [Modeling Others using Oneself in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1802.09640/) (2018)
- [Meta-Inverse Reinforcement Learning with Probabilistic Context Variables](https://mlsanity.com/paper/1909.09314/) (2019)
- [Learning a Prior over Intent via Meta-Inverse Reinforcement Learning](https://mlsanity.com/paper/1805.12573/) (2018)

## Built on by (most cited first, 30 of 39)

- [Large Language Models Fail on Trivial Alterations to Theory-of-Mind Tasks](https://mlsanity.com/paper/2302.08399/) (2023)
- [Understanding Social Reasoning in Language Models with Language Models](https://mlsanity.com/paper/2306.15448/) (2023)
- [MMToM-QA: Multimodal Theory of Mind Question Answering](https://mlsanity.com/paper/2401.08743/) (2024)
- [ToM2C: Target-oriented Multi-agent Communication and Cooperation with Theory of Mind](https://mlsanity.com/paper/2111.09189/) (2021)
- [MuMA-ToM: Multi-modal Multi-Agent Theory of Mind](https://mlsanity.com/paper/2408.12574/) (2024)
- [AGENT: A Benchmark for Core Psychological Reasoning](https://mlsanity.com/paper/2102.12321/) (2021)
- [Contrastive language and vision learning of general fashion concepts](https://mlsanity.com/paper/2204.03972/) (2022)
- [Benchmarks for Automated Commonsense Reasoning: A Survey](https://mlsanity.com/paper/2302.04752/) (2023)
- [Core Knowledge Deficits in Multi-Modal Language Models](https://mlsanity.com/paper/2410.10855/) (2024)
- [Towards A Holistic Landscape of Situated Theory of Mind in Large Language Models](https://mlsanity.com/paper/2310.19619/) (2023)
- [Language Models Represent Beliefs of Self and Others](https://mlsanity.com/paper/2402.18496/) (2024)
- [Theory of Mind in Large Language Models: Assessment and Enhancement](https://mlsanity.com/paper/2505.00026/) (2025)
- [NOPA: Neurally-guided Online Probabilistic Assistance for Building Socially Intelligent Home Assistants](https://mlsanity.com/paper/2301.05223/) (2023)
- [TimeToM: Temporal Space is the Key to Unlocking the Door of Large Language Models' Theory-of-Mind](https://mlsanity.com/paper/2407.01455/) (2024)
- [A Notion of Complexity for Theory of Mind via Discrete World Models](https://mlsanity.com/paper/2406.11911/) (2024)
- [SoMi-ToM: Evaluating Multi-Perspective Theory of Mind in Embodied Social Interactions](https://mlsanity.com/paper/2506.23046/) (2025)
- [EgoToM: Benchmarking Theory of Mind Reasoning from Egocentric Videos](https://mlsanity.com/paper/2503.22152/) (2025)
- [The Overcooked Generalisation Challenge: Evaluating Cooperation with Novel Partners in Unknown Environments Using Unsupervised Environment Design](https://mlsanity.com/paper/2406.17949/) (2024)
- [Limits of Theory of Mind Modelling in Dialogue-Based Collaborative Plan Acquisition](https://mlsanity.com/paper/2405.12621/) (2024)
- [EgoSocialArena: Benchmarking the Social Intelligence of Large Language Models from a First-person Perspective](https://mlsanity.com/paper/2410.06195/) (2024)
- [PLAICraft: Large-Scale Time-Aligned Vision-Speech-Action Dataset for Embodied AI](https://mlsanity.com/paper/2505.12707/) (2025)
- [Solving the Baby Intuitions Benchmark with a Hierarchically Bayesian Theory of Mind](https://mlsanity.com/paper/2208.02914/) (2022)
- [A Review on Machine Theory of Mind](https://mlsanity.com/paper/2303.11594/) (2023)
- [Comparing Machines and Children: Using Developmental Psychology Experiments to Assess the Strengths and Weaknesses of LaMDA Responses](https://mlsanity.com/paper/2305.11243/) (2023)
- [A Benchmark for Modeling Violation-of-Expectation in Physical Reasoning Across Event Categories](https://mlsanity.com/paper/2111.08826/) (2021)
- [Neural Reasoning About Agents' Goals, Preferences, and Actions](https://mlsanity.com/paper/2312.07122/) (2023)
- [Memory-Augmented Theory of Mind Network](https://mlsanity.com/paper/2301.06926/) (2023)
- [Explicit Modelling of Theory of Mind for Belief Prediction in Nonverbal Social Interactions](https://mlsanity.com/paper/2407.06762/) (2024)
- [Learning Theory of Mind via Dynamic Traits Attribution](https://mlsanity.com/paper/2204.09047/) (2022)
- [BDIQA: A New Dataset for Video Question Answering to Explore Cognitive Reasoning through Theory of Mind](https://mlsanity.com/paper/2402.07402/) (2024)

## Abstract (arXiv)

To achieve human-like common sense about everyday life, machine learning systems must understand and reason about the goals, preferences, and actions of other agents in the environment. By the end of their first year of life, human infants intuitively achieve such common sense, and these cognitive achievements lay the foundation for humans' rich and complex understanding of the mental states of others. Can machines achieve generalizable, commonsense reasoning about other agents like human infants? The Baby Intuitions Benchmark (BIB) challenges machines to predict the plausibility of an agent's behavior based on the underlying causes of its actions. Because BIB's content and paradigm are adopted from developmental cognitive science, BIB allows for direct comparison between human and machine performance. Nevertheless, recently proposed, deep-learning-based agency reasoning models fail to show infant-like reasoning, leaving BIB an open challenge.