# ChatCollab: Exploring Collaboration Between Humans and AI Agents in Software Teams

arXiv: https://arxiv.org/abs/2412.01992 · HTML: https://mlsanity.com/paper/2412.01992/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Dec 2024.

## Built on by (most cited first, 6 of 6)

- [A Survey on LLM-based Conversational User Simulation](https://mlsanity.com/paper/2604.24977/) (2026)
- [Security in the Age of AI Teammates: An Empirical Study of Agentic Pull Requests on GitHub](https://mlsanity.com/paper/2601.00477/) (2026)
- [Collaborative Document Editing with Multiple Users and AI Agents](https://mlsanity.com/paper/2509.11826/) (2025)
- [Evaluating the Effectiveness of Large Language Models in Solving Simple Programming Tasks: A User-Centered Study](https://mlsanity.com/paper/2507.04043/) (2025)
- [Agensh: Scaling Organizational Intelligence to 1,024 Agents](https://mlsanity.com/paper/2609.26781/) (2026)
- [SDAD: Spec-Driven Agentic Development for the AI-Native SDLC](https://mlsanity.com/paper/2608.20341/) (2026)

## Abstract (arXiv)

We explore the potential for productive team-based collaboration between humans and Artificial Intelligence (AI) by presenting and conducting initial tests with a general framework that enables multiple human and AI agents to work together as peers. ChatCollab's novel architecture allows agents - human or AI - to join collaborations in any role, autonomously engage in tasks and communication within Slack, and remain agnostic to whether their collaborators are human or AI. Using software engineering as a case study, we find that our AI agents successfully identify their roles and responsibilities, coordinate with other agents, and await requested inputs or deliverables before proceeding. In relation to three prior multi-agent AI systems for software development, we find ChatCollab AI agents produce comparable or better software in an interactive game development task. We also propose an automated method for analyzing collaboration dynamics that effectively identifies behavioral characteristics of agents with distinct roles, allowing us to quantitatively compare collaboration dynamics in a range of experimental conditions. For example, in comparing ChatCollab AI agents, we find that an AI CEO agent generally provides suggestions 2-4 times more often than an AI product manager or AI developer, suggesting agents within ChatCollab can meaningfully adopt differentiated collaborative roles. Our code and data can be found at: https://github.com/ChatCollab.