# Building Machines that Learn and Think with People

arXiv: https://arxiv.org/abs/2408.03943 · HTML: https://mlsanity.com/paper/2408.03943/

## Key facts

- Cited by 73 papers in the map (data through 14 Oct 2026).
- Builds on 87 of its 272 references that are in the map.
- First posted on arXiv 22 Jul 2024.

## Builds on (most cited first, 30 of 87)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [Do As I Can, Not As I Say: Grounding Language in Robotic Affordances](https://mlsanity.com/paper/2204.01691/) (2022)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [Improving Factuality and Reasoning in Language Models through Multiagent Debate](https://mlsanity.com/paper/2305.14325/) (2023)
- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Explanation in Artificial Intelligence: Insights from the Social Sciences](https://mlsanity.com/paper/1706.07269/) (2017)
- [ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks](https://mlsanity.com/paper/1912.01734/) (2019)
- [Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priors](https://mlsanity.com/paper/2203.13131/) (2022)
- [Cognitive Architectures for Language Agents](https://mlsanity.com/paper/2309.02427/) (2023)
- [Aligning Text-to-Image Models using Human Feedback](https://mlsanity.com/paper/2302.12192/) (2023)
- [PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Change](https://mlsanity.com/paper/2206.10498/) (2022)
- [Pyro: Deep Universal Probabilistic Programming](https://mlsanity.com/paper/1810.09538/) (2018)
- [On the Utility of Learning about Humans for Human-AI Coordination](https://mlsanity.com/paper/1910.05789/) (2019)
- [European Union regulations on algorithmic decision-making and a "right to explanation"](https://mlsanity.com/paper/1606.08813/) (2016)
- [Discovering Symbolic Models from Deep Learning with Inductive Biases](https://mlsanity.com/paper/2006.11287/) (2020)
- [Cooperative Inverse Reinforcement Learning](https://mlsanity.com/paper/1606.03137/) (2016)
- [Large Language Models Fail on Trivial Alterations to Theory-of-Mind Tasks](https://mlsanity.com/paper/2302.08399/) (2023)
- [Consistent Estimators for Learning to Defer to an Expert](https://mlsanity.com/paper/2006.01862/) (2020)
- [CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities](https://mlsanity.com/paper/2201.06796/) (2022)
- [Grounded Copilot: How Programmers Interact with Code-Generating Models](https://mlsanity.com/paper/2206.15000/) (2022)
- [Fine-tuning language models to find agreement among humans with diverse preferences](https://mlsanity.com/paper/2211.15006/) (2022)
- [Recasting Gradient-Based Meta-Learning as Hierarchical Bayes](https://mlsanity.com/paper/1801.08930/) (2018)
- [Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solve](https://mlsanity.com/paper/2309.13638/) (2023)
- [GitHub Copilot AI pair programmer: Asset or Liability?](https://mlsanity.com/paper/2206.15331/) (2022)

## Built on by (most cited first, 30 of 73)

- [Levels of Autonomy for AI Agents](https://mlsanity.com/paper/2506.12469/) (2025)
- [Human Creativity in the Age of LLMs: Randomized Experiments on Divergent and Convergent Thinking](https://mlsanity.com/paper/2410.03703/) (2024)
- [Cocoa: Co-Planning and Co-Execution with AI Agents](https://mlsanity.com/paper/2412.10999/) (2024)
- [Can Large Language Models Understand Symbolic Graphics Programs?](https://mlsanity.com/paper/2408.08313/) (2024)
- [Hypothesis-Driven Theory-of-Mind Reasoning for Large Language Models](https://mlsanity.com/paper/2502.11881/) (2025)
- [On Benchmarking Human-Like Intelligence in Machines](https://mlsanity.com/paper/2502.20502/) (2025)
- [Prototypical Human-AI Collaboration Behaviors from LLM-Assisted Writing in the Wild](https://mlsanity.com/paper/2505.16023/) (2025)
- [Assessing Adaptive World Models in Machines with Novel Games](https://mlsanity.com/paper/2507.12821/) (2025)
- [Collaborating with AI Agents: Field Experiments on Teamwork, Productivity, and Performance](https://mlsanity.com/paper/2503.18238/) (2025)
- [Modeling Open-World Cognition as On-Demand Synthesis of Probabilistic Models](https://mlsanity.com/paper/2507.12547/) (2025)
- [Imagining and building wise machines: The centrality of AI metacognition](https://mlsanity.com/paper/2411.02478/) (2024)
- [Measuring and mitigating overreliance to build human-compatible AI](https://mlsanity.com/paper/2509.08010/) (2025)
- [Data for Mathematical Copilots: Better Ways of Presenting Proofs for Machine Learning](https://mlsanity.com/paper/2412.15184/) (2024)
- [AI Assistance Reduces Persistence and Hurts Independent Performance](https://mlsanity.com/paper/2604.04721/) (2026)
- [Computational Hermeneutics: Evaluating generative AI as a cultural technology](https://mlsanity.com/paper/2604.16403/) (2026)
- [When Should We Orchestrate Multiple Agents?](https://mlsanity.com/paper/2503.13577/) (2025)
- [Learning User Simulators with Turing Rewards](https://mlsanity.com/paper/2606.19336/) (2026)
- [Revealing emergent human-like conceptual representations from language prediction](https://mlsanity.com/paper/2501.12547/) (2025)
- [What's Producible May Not Be Reachable: Measuring the Steerability of Generative Models](https://mlsanity.com/paper/2503.17482/) (2025)
- [Using LLMs to Advance the Cognitive Science of Collectives](https://mlsanity.com/paper/2506.00052/) (2025)
- [AI shares emotion with humans across languages and cultures](https://mlsanity.com/paper/2506.13978/) (2025)
- [Retrieval-augmented systems can be dangerous medical communicators](https://mlsanity.com/paper/2502.14898/) (2025)
- [Penalizing Transparency? How AI Disclosure and Author Demographics Shape Human and AI Judgments About Writing](https://mlsanity.com/paper/2507.01418/) (2025)
- [Multi-agent cooperation through learning-aware policy gradients](https://mlsanity.com/paper/2410.18636/) (2024)
- [Distributionally Robust Free Energy Principle for Decision-Making](https://mlsanity.com/paper/2503.13223/) (2025)
- [A large-scale evaluation of commonsense knowledge in humans and large language models](https://mlsanity.com/paper/2505.10309/) (2025)
- [Evaluating Language Models' Evaluations of Games](https://mlsanity.com/paper/2510.10930/) (2025)
- [A Matter of Interest: Understanding Interestingness of Math Problems in Humans and Language Models](https://mlsanity.com/paper/2511.08548/) (2025)
- [The Algorithmic State Architecture (ASA): An Integrated Framework for AI-Enabled Government](https://mlsanity.com/paper/2503.08725/) (2025)
- [Identifying, Evaluating, and Mitigating Risks of AI Thought Partnerships](https://mlsanity.com/paper/2505.16899/) (2025)

## Abstract (arXiv)

What do we want from machine intelligence? We envision machines that are not just tools for thought, but partners in thought: reasonable, insightful, knowledgeable, reliable, and trustworthy systems that think with us. Current artificial intelligence (AI) systems satisfy some of these criteria, some of the time. In this Perspective, we show how the science of collaborative cognition can be put to work to engineer systems that really can be called ``thought partners,'' systems built to meet our expectations and complement our limitations. We lay out several modes of collaborative thought in which humans and AI thought partners can engage and propose desiderata for human-compatible thought partnerships. Drawing on motifs from computational cognitive science, we motivate an alternative scaling path for the design of thought partners and ecosystems around their use through a Bayesian lens, whereby the partners we construct actively build and reason over models of the human and world.