mlsanityData to 7 Oct 2026

CODEX: A Cluster-Based Method for Explainable Reinforcement Learning

arXiv 2312.04216 ↗First posted 7 Dec 2023cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

Despite the impressive feats demonstrated by Reinforcement Learning (RL), these algorithms have seen little adoption in high-risk, real-world applications due to current difficulties in explaining RL agent actions and building user trust. We present Counterfactual Demonstrations for Explanation (CODEX), a method that incorporates semantic clustering, which can effectively summarize RL agent behavior in the state-action space. Experimentation on the MiniGrid and StarCraft II gaming environments reveals the semantic clusters retain temporal as well as entity information, which is reflected in the constructed summary of agent behavior. Furthermore, clustering the discrete+continuous game-state latent representations identifies the most crucial episodic events, demonstrating a relationship between the latent and semantic spaces. This work contributes to the growing body of work that strives to unlock the power of RL for widespread use by leveraging and extending techniques from Natural Language Processing.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 21 of its 41 references that are in the map. First posted on arXiv 7 Dec 2023.

Builds on

21 of 41 references.

References · 21

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  4. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  5. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  6. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  7. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  8. 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,543
  9. 2017Learning Important Features Through Propagating Activation Differencescited by 1,454
  10. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  11. 2020Mastering Atari with Discrete World Modelscited by 990
  12. 2019Evaluating the Factual Consistency of Abstractive Text Summarizationcited by 578
9 more
  1. 2019Model-Based Reinforcement Learning for Ataricited by 495
  2. 2020BERTweet: A pre-trained language model for English Tweetscited by 346
  3. 2017StarCraft II: A New Challenge for Reinforcement Learningcited by 268
  4. 2020The Computational Limits of Deep Learningcited by 188
  5. 2020Explainable Reinforcement Learning: A Surveycited by 139
  6. 2020Explainability in Deep Reinforcement Learningcited by 96
  7. 2021Online and Offline Reinforcement Learning by Planning with a Learned Modelcited by 90
  8. 2018Contrastive Explanations for Reinforcement Learning in terms of Expected Consequencescited by 44
  9. 2013On the Complexity of Solving Markov Decision Problemscited by 31

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025Unveiling the Black Box: A Multi-Layer Framework for Explaining Reinforcement Learning-Based Cyber Agentscited by 4
  2. 2026SAT-RTS: A systematic framework for tactical knowledge extraction and visualization-based analysis in real-time strategy gamescited by 0
  3. 2026Hierarchical Reinforcement Learning in StarCraft Micromanagement with Influence Maps and Cluster-based Scriptscited by 0