# Human Thinking under Plural LLM Assistance: Mathematical Problem Solving and Open-Ended Writing

arXiv: https://arxiv.org/abs/2604.02677 · HTML: https://mlsanity.com/paper/2604.02677/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 12 of its 57 references that are in the map.
- First posted on arXiv 3 Apr 2026.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 12 of 12)

- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society](https://mlsanity.com/paper/2303.17760/) (2023)
- [CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities](https://mlsanity.com/paper/2201.06796/) (2022)
- [Does Writing with Language Models Reduce Content Diversity?](https://mlsanity.com/paper/2309.05196/) (2023)
- [Homogenization Effects of Large Language Models on Human Creative Ideation](https://mlsanity.com/paper/2402.01536/) (2024)
- [The Assistant Axis: Situating and Stabilizing the Default Persona of Language Models](https://mlsanity.com/paper/2601.10387/) (2026)
- [The Metacognitive Demands and Opportunities of Generative AI](https://mlsanity.com/paper/2312.10893/) (2023)
- ["I'm Not Sure, But...": Examining the Impact of Large Language Models' Uncertainty Expression on User Reliance and Trust](https://mlsanity.com/paper/2405.00623/) (2024)
- [Human Creativity in the Age of LLMs: Randomized Experiments on Divergent and Convergent Thinking](https://mlsanity.com/paper/2410.03703/) (2024)
- [Fostering Appropriate Reliance on Large Language Models: The Role of Explanations, Sources, and Inconsistencies](https://mlsanity.com/paper/2502.08554/) (2025)
- [Impact of Guidance and Interaction Strategies for LLM Use on Learner Performance and Perception](https://mlsanity.com/paper/2310.13712/) (2023)
- [A Taxonomy of Linguistic Expressions That Contribute To Anthropomorphism of Language Technologies](https://mlsanity.com/paper/2502.09870/) (2025)

## Abstract (arXiv)

Large language models are changing not only the kind of assistance people receive, but also how that assistance is organized. Instead of working with a single general-purpose chatbot, people can now receive help from systems arranged as peers, specialists, or multiple agents with distinct roles. However, it remains unclear how these forms of plural LLM assistance affect human performance, confidence, and diversity of thought. We conducted two controlled experiments involving 562 participants to examine the effects of using multiple LLMs on mathematical problem-solving and writing. In a math task, participants worked with no LLM, an expert assistant, peer-like agents that surfaced common errors, or both an expert and a peer-like assistant. The expert-plus-peer condition produced the strongest unassisted post-task performance. In a writing task, participants wrote with no LLM, a single generalist assistant, or a pair of role-specialized assistants. LLM assistance improved essay quality, but the role-specialized pair preserved greater idea diversity than the single assistant. Together, these findings identify the arrangement of LLM assistance as a consequential design variable for human-AI collaboration.