# MARLeME: A Multi-Agent Reinforcement Learning Model Extraction Library

arXiv: https://arxiv.org/abs/2004.07928 · HTML: https://mlsanity.com/paper/2004.07928/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Apr 2020.

## Built on by (most cited first, 7 of 7)

- [Explainable Reinforcement Learning for Broad-XAI: A Conceptual Framework and Survey](https://mlsanity.com/paper/2108.09003/) (2021)
- [MAVIPER: Learning Decision Tree Policies for Interpretable Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2205.12449/) (2022)
- [Toward Policy Explanations for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2204.12568/) (2022)
- [Multi-Agent Reinforcement Learning: Methods, Applications, Visionary Prospects, and Challenges](https://mlsanity.com/paper/2305.10091/) (2023)
- [VidModEx: Interpretable and Efficient Black Box Model Extraction for High-Dimensional Spaces](https://mlsanity.com/paper/2408.02140/) (2024)
- [Argumentation and Machine Learning](https://mlsanity.com/paper/2410.23724/) (2024)
- [A MARL-based Approach for Easing MAS Organization Engineering](https://mlsanity.com/paper/2506.05437/) (2025)

## Abstract (arXiv)

Multi-Agent Reinforcement Learning (MARL) encompasses a powerful class of methodologies that have been applied in a wide range of fields. An effective way to further empower these methodologies is to develop libraries and tools that could expand their interpretability and explainability. In this work, we introduce MARLeME: a MARL model extraction library, designed to improve explainability of MARL systems by approximating them with symbolic models. Symbolic models offer a high degree of interpretability, well-defined properties, and verifiable behaviour. Consequently, they can be used to inspect and better understand the underlying MARL system and corresponding MARL agents, as well as to replace all/some of the agents that are particularly safety and security critical.