# Structure Matters: Evaluating Multi-Agents Orchestration in Generative Therapeutic Chatbots

arXiv: https://arxiv.org/abs/2603.00774 · HTML: https://mlsanity.com/paper/2603.00774/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Medical and scientific domain agents, one of 16 directions in LLM agents and tool use.
- Builds on 21 of its 46 references that are in the map.
- First posted on arXiv 28 Feb 2026.

## Placed in

- LLM agents and tool use / [Medical and scientific domain agents](https://mlsanity.com/agents/medical-and-scientific-domain-agents/)

## Builds on (most cited first, 21 of 21)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Secrets of RLHF in Large Language Models Part I: PPO](https://mlsanity.com/paper/2307.04964/) (2023)
- [Limitations of the LLM-as-a-Judge Approach for Evaluating LLM Outputs in Expert Knowledge Tasks](https://mlsanity.com/paper/2410.20266/) (2024)
- [Facilitating Self-Guided Mental Health Interventions Through Human-Language Model Interaction: A Case Study of Cognitive Restructuring](https://mlsanity.com/paper/2310.15461/) (2023)
- [MindfulDiary: Harnessing Large Language Model to Support Psychiatric Patients' Journaling](https://mlsanity.com/paper/2310.05231/) (2023)
- [ChaCha: Leveraging Large Language Models to Prompt Children to Share Their Emotions about Personal Events](https://mlsanity.com/paper/2309.12244/) (2023)
- [The Typing Cure: Experiences with Large Language Model Chatbots for Mental Health Support](https://mlsanity.com/paper/2401.14362/) (2024)
- [MetaAgent: Automatically Constructing Multi-Agent Systems Based on Finite State Machines](https://mlsanity.com/paper/2507.22606/) (2025)
- [EmoAgent: Assessing and Safeguarding Human-AI Interaction for Mental Health Safety](https://mlsanity.com/paper/2504.09689/) (2025)
- [An Evaluation of Generative Pre-Training Model-based Therapy Chatbot for Caregivers](https://mlsanity.com/paper/2107.13115/) (2021)
- [PersianLLaMA: Towards Building First Persian Large Language Model](https://mlsanity.com/paper/2312.15713/) (2023)
- [Private Yet Social: How LLM Chatbots Support and Challenge Eating Disorder Recovery](https://mlsanity.com/paper/2412.11656/) (2024)
- [HamRaz: A Culture-Based Persian Conversation Dataset for Person-Centered Therapy Using LLM Agents](https://mlsanity.com/paper/2502.05982/) (2025)
- [An Empathetic AI Coach for Self-Attachment Therapy](https://mlsanity.com/paper/2209.08316/) (2022)
- [Comparing the Efficacy of GPT-4 and Chat-GPT in Mental Health Care: A Blind Assessment of Large Language Models for Psychological Support](https://mlsanity.com/paper/2405.09300/) (2024)
- [AgentMental: An Interactive Multi-Agent Framework for Explainable and Adaptive Mental Health Assessment](https://mlsanity.com/paper/2508.11567/) (2025)
- [Can a Funny Chatbot Make a Difference? Infusing Humor into Conversational Agent for Behavioral Intervention](https://mlsanity.com/paper/2403.00365/) (2024)
- [A Deep Learning Based Chatbot for Campus Psychological Therapy](https://mlsanity.com/paper/1910.06707/) (2019)
- [From Words and Exercises to Wellness: Farsi Chatbot for Self-Attachment Technique](https://mlsanity.com/paper/2310.09362/) (2023)
- [A Multilingual Virtual Guide for Self-Attachment Technique](https://mlsanity.com/paper/2310.18366/) (2023)

## Abstract (arXiv)

While large language models (LLMs) excel at open-ended dialogue, effective psychotherapy requires structured progression and adherence to clinical protocols, making the design of psychotherapist chatbots challenging. We investigate how different LLM-based designs shape perceived therapeutic dialogue in a chatbot grounded in the Self-Attachment Technique (SAT), a novel self-administered psychotherapy rooted in attachment theory. We compare three architectural variants: (1) a multi-agent system utilizing finite state machine aligned with therapeutic stages and a shared long-term memory, (2) a single-agent using identical knowledge-base and the same prompts, and (3) an unguided LLM. In an eight-day randomized controlled trial (RCT) with N=66 Farsi-speaking participants, balanced across the three chatbots, the multi-agent system is perceived as significantly more natural and human-like than the other variants and achieves higher ratings across most other metrics. These findings demonstrate that for therapeutic AI, architectural orchestration is as critical as prompt engineering in fostering natural, engaging dialogue.