# Towards Ethical Multi-Agent Systems of Large Language Models: A Mechanistic Interpretability Perspective

arXiv: https://arxiv.org/abs/2512.04691 · HTML: https://mlsanity.com/paper/2512.04691/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 27 of its 41 references that are in the map.
- First posted on arXiv 4 Dec 2025.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)
- Mechanistic interpretability / [Knowledge and representations inside LLMs](https://mlsanity.com/mechinterp/knowledge-and-representations-inside-llms/)

## Builds on (most cited first, 27 of 27)

- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [Representation Engineering: A Top-Down Approach to AI Transparency](https://mlsanity.com/paper/2310.01405/) (2023)
- [RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models](https://mlsanity.com/paper/2009.11462/) (2020)
- [Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!](https://mlsanity.com/paper/2310.03693/) (2023)
- [Steering Llama 2 via Contrastive Activation Addition](https://mlsanity.com/paper/2312.06681/) (2023)
- [In-context Learning and Induction Heads](https://mlsanity.com/paper/2209.11895/) (2022)
- [Steering Language Models With Activation Engineering](https://mlsanity.com/paper/2308.10248/) (2023)
- [Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models](https://mlsanity.com/paper/2403.19647/) (2024)
- [Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretability](https://mlsanity.com/paper/2301.04709/) (2023)
- [LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70B](https://mlsanity.com/paper/2310.20624/) (2023)
- [MultiAgentBench: Evaluating the Collaboration and Competition of LLM agents](https://mlsanity.com/paper/2503.01935/) (2025)
- [Multi-Agent Risks from Advanced AI](https://mlsanity.com/paper/2502.14143/) (2025)
- [Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systems](https://mlsanity.com/paper/2504.01990/) (2025)
- [Sustainable Modular Debiasing of Language Models](https://mlsanity.com/paper/2109.03646/) (2021)
- [Self-Alignment of Large Language Models via Monopolylogue-based Social Scene Simulation](https://mlsanity.com/paper/2402.05699/) (2024)
- [GUARDIAN: Safeguarding LLM Multi-Agent Collaborations with Temporal Graph Modeling](https://mlsanity.com/paper/2505.19234/) (2025)
- [The Traitors: Deception and Trust in Multi-Agent Language Model Simulations](https://mlsanity.com/paper/2505.12923/) (2025)
- [KG4Diagnosis: A Hierarchical Multi-Agent LLM Framework with Knowledge Graph Enhancement for Medical Diagnosis](https://mlsanity.com/paper/2412.16833/) (2024)
- [Large Language Models Miss the Multi-Agent Mark](https://mlsanity.com/paper/2505.21298/) (2025)
- [Can LLMs Recognize Toxicity? A Structured Investigation Framework and Toxicity Metric](https://mlsanity.com/paper/2402.06900/) (2024)
- [Group Think: Multiple Concurrent Reasoning Agents Collaborating at Token Level Granularity](https://mlsanity.com/paper/2505.11107/) (2025)
- [A Tale of Pronouns: Interpretability Informs Gender Bias Mitigation for Fairer Instruction-Tuned Machine Translation](https://mlsanity.com/paper/2310.12127/) (2023)
- [Robustly Improving LLM Fairness in Realistic Settings via Interpretability](https://mlsanity.com/paper/2506.10922/) (2025)
- [MAEBE: Multi-Agent Emergent Behavior Framework](https://mlsanity.com/paper/2506.03053/) (2025)
- [Orchestrating Human-AI Teams: The Manager Agent as a Unifying Research Challenge](https://mlsanity.com/paper/2510.02557/) (2025)
- [Prefix Guidance: A Steering Wheel for Large Language Models to Defend Against Jailbreak Attacks](https://mlsanity.com/paper/2408.08924/) (2024)
- [Integrating Counterfactual Simulations with Language Models for Explaining Multi-Agent Behaviour](https://mlsanity.com/paper/2505.17801/) (2025)

## Built on by (most cited first, 1 of 1)

- [Multi-Agent Systems are Mixtures of Experts: Who Becomes an Influencer?](https://mlsanity.com/paper/2605.25929/) (2026)

## Abstract (arXiv)

Large language models (LLMs) have been widely deployed in various applications, often functioning as autonomous agents that interact with each other in multi-agent systems. While these systems have shown promise in enhancing capabilities and enabling complex tasks, they also pose significant ethical challenges. This position paper outlines a research agenda aimed at ensuring the ethical behavior of multi-agent systems of LLMs (MALMs) from the perspective of mechanistic interpretability. We identify three key research challenges: (i) developing comprehensive evaluation frameworks to assess ethical behavior at individual, interactional, and systemic levels; (ii) elucidating the internal mechanisms that give rise to emergent behaviors through mechanistic interpretability; and (iii) implementing targeted parameter-efficient alignment techniques to steer MALMs towards ethical behaviors without compromising their performance.