# The Neuro-Symbolic Inverse Planning Engine (NIPE): Modeling Probabilistic Social Inferences from Linguistic Inputs

arXiv: https://arxiv.org/abs/2306.14325 · HTML: https://mlsanity.com/paper/2306.14325/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 52 references that are in the map.
- First posted on arXiv 25 Jun 2023.

## Builds on (most cited first, 23 of 23)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [LLM+P: Empowering Large Language Models with Optimal Planning Proficiency](https://mlsanity.com/paper/2304.11477/) (2023)
- [On the Utility of Learning about Humans for Human-AI Coordination](https://mlsanity.com/paper/1910.05789/) (2019)
- [Large Language Models Fail on Trivial Alterations to Theory-of-Mind Tasks](https://mlsanity.com/paper/2302.08399/) (2023)
- [Faithful Chain-of-Thought Reasoning](https://mlsanity.com/paper/2301.13379/) (2023)
- [Consistent Estimators for Learning to Defer to an Expert](https://mlsanity.com/paper/2006.01862/) (2020)
- [Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Models](https://mlsanity.com/paper/2305.14763/) (2023)
- [Synchromesh: Reliable code generation from pre-trained language models](https://mlsanity.com/paper/2201.11227/) (2022)
- [Generalized Planning in PDDL Domains with Pretrained Large Language Models](https://mlsanity.com/paper/2305.11014/) (2023)
- [Learning to Complement Humans](https://mlsanity.com/paper/2005.00582/) (2020)
- [From Word Models to World Models: Translating from Natural Language to the Probabilistic Language of Thought](https://mlsanity.com/paper/2306.12672/) (2023)
- [Sequential Monte Carlo Steering of Large Language Models using Probabilistic Programs](https://mlsanity.com/paper/2306.03081/) (2023)
- [Online Bayesian Goal Inference for Boundedly-Rational Planning Agents](https://mlsanity.com/paper/2006.07532/) (2020)
- [Structured, flexible, and robust: benchmarking and improving large language models towards more human-like behavior in out-of-distribution reasoning tasks](https://mlsanity.com/paper/2205.05718/) (2022)
- [On the Feasibility of Learning, Rather than Assuming, Human Biases for Reward Inference](https://mlsanity.com/paper/1906.09624/) (2019)
- [Learning the Preferences of Ignorant, Inconsistent Agents](https://mlsanity.com/paper/1512.05832/) (2015)
- [AGENT: A Benchmark for Core Psychological Reasoning](https://mlsanity.com/paper/2102.12321/) (2021)
- [Dialectical language model evaluation: An initial appraisal of the commonsense spatial reasoning abilities of LLMs](https://mlsanity.com/paper/2304.11164/) (2023)
- [Modeling the Mistakes of Boundedly Rational Agents Within a Bayesian Theory of Mind](https://mlsanity.com/paper/2106.13249/) (2021)
- [LaMPP: Language Models as Probabilistic Priors for Perception and Action](https://mlsanity.com/paper/2302.02801/) (2023)

## Built on by (most cited first, 23 of 23)

- [Building Machines that Learn and Think with People](https://mlsanity.com/paper/2408.03943/) (2024)
- [OpenToM: A Comprehensive Benchmark for Evaluating Theory-of-Mind Reasoning Capabilities of Large Language Models](https://mlsanity.com/paper/2402.06044/) (2024)
- [Syntactic and Semantic Control of Large Language Models via Sequential Monte Carlo](https://mlsanity.com/paper/2504.13139/) (2025)
- [LLMs as Planning Formalizers: A Survey for Leveraging Large Language Models to Construct Automated Planning Models](https://mlsanity.com/paper/2503.18971/) (2025)
- [Hypothesis-Driven Theory-of-Mind Reasoning for Large Language Models](https://mlsanity.com/paper/2502.11881/) (2025)
- [Pragmatic Instruction Following and Goal Assistance via Cooperative Language-Guided Inverse Planning](https://mlsanity.com/paper/2402.17930/) (2024)
- [GOMA: Proactive Embodied Cooperative Communication via Goal-Oriented Mental Alignment](https://mlsanity.com/paper/2403.11075/) (2024)
- [Social World Models](https://mlsanity.com/paper/2509.00559/) (2025)
- [Modeling Open-World Cognition as On-Demand Synthesis of Probabilistic Models](https://mlsanity.com/paper/2507.12547/) (2025)
- [Neuro-Symbolic AI: Explainability, Challenges, and Future Trends](https://mlsanity.com/paper/2411.04383/) (2024)
- [Grounding Language about Belief in a Bayesian Theory-of-Mind](https://mlsanity.com/paper/2402.10416/) (2024)
- [Resource Rational Contractualism Should Guide AI Alignment](https://mlsanity.com/paper/2506.17434/) (2025)
- [Infusing Theory of Mind into Socially Intelligent LLM Agents](https://mlsanity.com/paper/2509.22887/) (2025)
- [Understanding Epistemic Language with a Language-augmented Bayesian Theory of Mind](https://mlsanity.com/paper/2408.12022/) (2024)
- [EnigmaToM: Improve LLMs' Theory-of-Mind Reasoning Capabilities with Neural Knowledge Base of Entity States](https://mlsanity.com/paper/2503.03340/) (2025)
- [Overcoming Multi-step Complexity in Multimodal Theory-of-Mind Reasoning: A Scalable Bayesian Planner](https://mlsanity.com/paper/2506.01301/) (2025)
- [Post-training makes large language models less human-like](https://mlsanity.com/paper/2605.07632/) (2026)
- [Think Thrice Before You Speak: Dual knowledge-enhanced Theory-of-Mind Reasoning for Persuasive Agents](https://mlsanity.com/paper/2605.22602/) (2026)
- [Grounding Social Perception in Intuitive Physics](https://mlsanity.com/paper/2603.27410/) (2026)
- [Bridging the Gap: Representation Spaces in Neuro-Symbolic AI](https://mlsanity.com/paper/2411.04393/) (2024)
- [Belief Attribution as Mental Explanation: The Role of Accuracy, Informativity, and Causality](https://mlsanity.com/paper/2505.19376/) (2025)
- [What's in the Box? Reasoning about Unseen Objects from Multimodal Cues](https://mlsanity.com/paper/2506.14212/) (2025)
- [MindZero: Learning Online Mental Reasoning With Zero Annotations](https://mlsanity.com/paper/2606.00240/) (2026)

## Abstract (arXiv)

Human beings are social creatures. We routinely reason about other agents, and a crucial component of this social reasoning is inferring people's goals as we learn about their actions. In many settings, we can perform intuitive but reliable goal inference from language descriptions of agents, actions, and the background environments. In this paper, we study this process of language driving and influencing social reasoning in a probabilistic goal inference domain. We propose a neuro-symbolic model that carries out goal inference from linguistic inputs of agent scenarios. The "neuro" part is a large language model (LLM) that translates language descriptions to code representations, and the "symbolic" part is a Bayesian inverse planning engine. To test our model, we design and run a human experiment on a linguistic goal inference task. Our model closely matches human response patterns and better predicts human judgements than using an LLM alone.