# K-SHAP: Policy Clustering Algorithm for Anonymous Multi-Agent State-Action Pairs

arXiv: https://arxiv.org/abs/2302.11996 · HTML: https://mlsanity.com/paper/2302.11996/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 58 references that are in the map.
- First posted on arXiv 23 Feb 2023.

## Builds on (most cited first, 21 of 21)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [Recurrent World Models Facilitate Policy Evolution](https://mlsanity.com/paper/1809.01999/) (2018)
- [Consistent Individualized Feature Attribution for Tree Ensembles](https://mlsanity.com/paper/1802.03888/) (2018)
- [Towards K-means-friendly Spaces: Simultaneous Deep Learning and Clustering](https://mlsanity.com/paper/1610.04794/) (2016)
- [TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors](https://mlsanity.com/paper/2101.06557/) (2021)
- [Multi-Agent Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1807.09936/) (2018)
- [ClusterGAN : Latent Space Clustering in Generative Adversarial Networks](https://mlsanity.com/paper/1809.03627/) (2018)
- [Identifiability in inverse reinforcement learning](https://mlsanity.com/paper/2106.03498/) (2021)
- [Get Real: Realism Metrics for Robust Limit Order Book Market Simulations](https://mlsanity.com/paper/1912.04941/) (2019)
- [ABIDES-Gym: Gym Environments for Multi-Agent Discrete Event Simulation and Application to Financial Markets](https://mlsanity.com/paper/2110.14771/) (2021)
- [ABIDES: Towards High-Fidelity Market Simulation for AI Research](https://mlsanity.com/paper/1904.12066/) (2019)
- [Towards Realistic Market Simulations: a Generative Adversarial Networks Approach](https://mlsanity.com/paper/2110.13287/) (2021)
- [Learning to simulate realistic limit order book markets from data as a World Agent](https://mlsanity.com/paper/2210.09897/) (2022)
- [Top-Down Deep Clustering with Multi-generator GANs](https://mlsanity.com/paper/2112.03398/) (2021)
- [Limited or Biased: Modeling Sub-Rational Human Investors in Financial Markets](https://mlsanity.com/paper/2210.08569/) (2022)
- [Equitable Marketplace Mechanism Design](https://mlsanity.com/paper/2209.15418/) (2022)

## Built on by (most cited first, 6 of 6)

- [On the Constrained Time-Series Generation Problem](https://mlsanity.com/paper/2307.01717/) (2023)
- [Conditional Generators for Limit Order Book Environments: Explainability, Challenges, and Robustness](https://mlsanity.com/paper/2306.12806/) (2023)
- [LLM-driven Imitation of Subrational Behavior : Illusion or Reality?](https://mlsanity.com/paper/2402.08755/) (2024)
- [SurvBeX: An explanation method of the machine learning survival models based on the Beran estimator](https://mlsanity.com/paper/2308.03730/) (2023)
- [Measuring Cross-Task Behavioral Consistency in Language Model Agents](https://mlsanity.com/paper/2608.13598/) (2026)
- [SurvBeNIM: The Beran-Based Neural Importance Model for Explaining the Survival Models](https://mlsanity.com/paper/2312.06638/) (2023)

## Abstract (arXiv)

Learning agent behaviors from observational data has shown to improve our understanding of their decision-making processes, advancing our ability to explain their interactions with the environment and other agents. While multiple learning techniques have been proposed in the literature, there is one particular setting that has not been explored yet: multi agent systems where agent identities remain anonymous. For instance, in financial markets labeled data that identifies market participant strategies is typically proprietary, and only the anonymous state-action pairs that result from the interaction of multiple market participants are publicly available. As a result, sequences of agent actions are not observable, restricting the applicability of existing work. In this paper, we propose a Policy Clustering algorithm, called K-SHAP, that learns to group anonymous state-action pairs according to the agent policies. We frame the problem as an Imitation Learning (IL) task, and we learn a world-policy able to mimic all the agent behaviors upon different environmental states. We leverage the world-policy to explain each anonymous observation through an additive feature attribution method called SHAP (SHapley Additive exPlanations). Finally, by clustering the explanations we show that we are able to identify different agent policies and group observations accordingly. We evaluate our approach on simulated synthetic market data and a real-world financial dataset. We show that our proposal significantly and consistently outperforms the existing methods, identifying different agent strategies.