# Multi-Modal Multi-Agent Robotic Cognitive Alignment enabled by Non-Invasive Consumer Brain Computer Interfaces: A Proof of Concept Exploration

arXiv: https://arxiv.org/abs/2606.13190 · HTML: https://mlsanity.com/paper/2606.13190/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use.
- Builds on 10 of its 36 references that are in the map.
- First posted on arXiv 11 Jun 2026.

## Placed in

- LLM agents and tool use / [Multimodal and embodied agents](https://mlsanity.com/agents/multimodal-and-embodied-agents/)

## Builds on (most cited first, 10 of 10)

- [A Survey on Large Language Model based Autonomous Agents](https://mlsanity.com/paper/2308.11432/) (2023)
- [Large Language Model based Multi-Agents: A Survey of Progress and Challenges](https://mlsanity.com/paper/2402.01680/) (2024)
- [Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task](https://mlsanity.com/paper/2506.08872/) (2025)
- [Embodied AI Agents: Modeling the World](https://mlsanity.com/paper/2506.22355/) (2025)
- [NeuroChat: A Neuroadaptive AI Chatbot for Customizing Learning Experiences](https://mlsanity.com/paper/2503.07599/) (2025)
- [United Minds or Isolated Agents? Exploring Coordination of LLMs under Cognitive Load Theory](https://mlsanity.com/paper/2506.06843/) (2025)
- [EMOTION: Expressive Motion Sequence Generation for Humanoid Robots with In-Context Learning](https://mlsanity.com/paper/2410.23234/) (2024)
- [Short-Form Videos Degrade Our Capacity to Retain Intentions: Effect of Context Switching On Prospective Memory](https://mlsanity.com/paper/2302.03714/) (2023)
- [Learning to Interrupt in Language-based Multi-agent Communication](https://mlsanity.com/paper/2604.06452/) (2026)
- [ELEGNT: Expressive and Functional Movement Design for Non-anthropomorphic Robot](https://mlsanity.com/paper/2501.12493/) (2025)

## Abstract (arXiv)

While non-verbal behaviors and expressive movements are essential for natural human-robot interaction, existing methods often overlook a crucial element: the human's internal cognitive state. Frequently, proactive multi-agent systems can interrupt humans at inopportune moments, leading to cognitive overload and decreased task performance. This paper introduces a framework for generating "cognitively aligned" multi-agent interactions, enhancing the ability of robotic systems to contextually defer communications to the user of an agent system during moments of high human mental workload and engagement. We present the design and implementation of a closed-loop architecture that explores the interplay between autonomous task execution and real-time neurophysiological focus. Using a consumer-grade Brain-Computer Interface (BCI), our approach continuously monitors Electroencephalography (EEG) spectral band powers while a human performs an engagement-inducing task. We propose an engagement-driven pipeline where an HTTP-based signaling mechanism places a primary agent's sensory inputs and audio outputs into a holding state upon detecting high engagement. This allows secondary agents to seamlessly process complex, delegated tasks in the background. Once the human's cognitive state returns to a lower cognitive load baseline, the primary agent releases the queued agent message. Our preliminary results demonstrate the feasibility of leveraging real-time signal processing, Large Language Models (LLMs), and physical robotic embodiments to create cognitively-aware, non-intrusive multi-agent systems.