mlsanityData to 14 Oct 2026

Enhancing Interpretability in Deep Reinforcement Learning through Semantic Clustering

arXiv 2409.17411 ↗First posted 25 Sep 2024cs.AIcited by 1

On the map: Deep reinforcement learning methods → Interpretable and logic-guided RL

Abstract (arXiv)

In this paper, we explore semantic clustering properties of deep reinforcement learning (DRL) to improve its interpretability and deepen our understanding of its internal semantic organization. In this context, semantic clustering refers to the ability of neural networks to cluster inputs based on their semantic similarity in the feature space. We propose a DRL architecture that incorporates a novel semantic clustering module that combines feature dimensionality reduction with online clustering. This module integrates seamlessly into the DRL training pipeline, addressing the instability of t-SNE and eliminating the need for extensive manual annotation inherent to prior semantic analysis methods. We experimentally validate the effectiveness of the proposed module and demonstrate its ability to reveal semantic clustering properties within DRL. Furthermore, we introduce new analytical methods based on these properties to provide insights into the hierarchical structure of policies and semantic organization within the feature space. Our code is available at https://github.com/ualiangzhang/semantic_rl.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Interpretable and logic-guided RL, one of 15 directions in Deep reinforcement learning methods. Builds on 24 of its 44 references that are in the map. First posted on arXiv 25 Sep 2024.

Builds on

24 of 44 references.

References · 24

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2017Neural Discrete Representation Learningcited by 4,712
  3. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,387
  4. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 883
  5. 2015Unsupervised Deep Embedding for Clustering Analysiscited by 767
  6. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
  7. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 658
  8. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 483
  9. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 436
  10. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 253
  11. 2018An Introduction to Deep Reinforcement Learningcited by 206
  12. 2021Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learningcited by 123
12 more
  1. 2023Re-IQA: Unsupervised Learning for Image Quality Assessment in the Wildcited by 97
  2. 2019Towards Interpretable Reinforcement Learning Using Attention Augmented Agentscited by 82
  3. 2018SDRL: Interpretable and Data-efficient Deep Reinforcement Learning Leveraging Symbolic Planningcited by 78
  4. 2016Graying the black box: Understanding DQNscited by 68
  5. 2014word2vec Parameter Learning Explainedcited by 53
  6. 2020Improving Unsupervised Image Clustering With Robust Learningcited by 51
  7. 2019Low Bit-Rate Speech Coding with VQ-VAE and a WaveNet Decodercited by 49
  8. 2021Towards Robust Bisimulation Metric Learningcited by 49
  9. 2021Vector Quantized Models for Planningcited by 48
  10. 2017Discriminatively Boosted Image Clustering with Fully Convolutional Auto-Encoderscited by 32
  11. 2023Concept Learning for Interpretable Multi-Agent Reinforcement Learningcited by 19
  12. 2020Smaller World Models for Reinforcement Learningcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026SAT-RTS: A systematic framework for tactical knowledge extraction and visualization-based analysis in real-time strategy gamescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Towards Automated Semantic Interpretability in Reinforcement Learning via Vision-Language Models
  2. 2022Deep Reinforcement Learning with Vector Quantized Encoding
  3. 2024Discovering Behavioral Modes in Deep Reinforcement Learning Policies Using Trajectory Clustering in Latent Space
  4. 2025Compositional Concept-Based Neuron-Level Interpretability for Deep Reinforcement Learning
  5. 2024Interpretable Modeling of Deep Reinforcement Learning Driven Scheduling
  6. 2025Policy-Based Trajectory Clustering in Offline Reinforcement Learning
  7. 2025A Survey on Explainable Deep Reinforcement Learning
  8. 2026Boosting Deep Reinforcement Learning with Semantic Knowledge for Robotic Manipulators