mlsanityData to 14 Oct 2026

Surrogate Fitness Metrics for Interpretable Reinforcement Learning

arXiv 2504.14645 ↗First posted 20 Apr 2025cs.LG cs.AIcited by 1

On the map: Deep reinforcement learning methods → Interpretable and logic-guided RL

Abstract (arXiv)

We employ an evolutionary optimization framework that perturbs initial states to generate informative and diverse policy demonstrations. A joint surrogate fitness function guides the optimization by combining local diversity, behavioral certainty, and global population diversity. To assess demonstration quality, we apply a set of evaluation metrics, including the reward-based optimality gap, fidelity interquartile means (IQMs), fitness composition analysis, and trajectory visualizations. Hyperparameter sensitivity is also examined to better understand the dynamics of trajectory optimization. Our findings demonstrate that optimizing trajectory selection via surrogate fitness metrics significantly improves interpretability of RL policies in both discrete and continuous environments. In gridworld domains, evaluations reveal significantly enhanced demonstration fidelities compared to random and ablated baselines. In continuous control, the proposed framework offers valuable insights, particularly for early-stage policies, while fidelity-based optimization proves more effective for mature policies. By refining and systematically analyzing surrogate fitness functions, this study advances the interpretability of RL models. The proposed improvements provide deeper insights into RL decision-making, benefiting applications in safety-critical and explainability-focused domains.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Interpretable and logic-guided RL, one of 15 directions in Deep reinforcement learning methods. Builds on 25 of its 44 references that are in the map. First posted on arXiv 20 Apr 2025.

Builds on

25 of 44 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024REACT: Revealing Evolutionary Action Consequence Trajectories for Interpretable Reinforcement Learning4 Apr

Other references · 24

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  3. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  4. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  5. 2017Understanding Black-box Predictions via Influence Functionscited by 1,741
  6. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
  7. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 483
  8. 2020Identifying Mislabeled Data using the Area Under the Margin Rankingcited by 261
  9. 2022Evolving Curricula with Regret-Based Environment Designcited by 173
  10. 2020Explainable Reinforcement Learning: A Surveycited by 141
  11. 2020Effective Diversity in Population Based Reinforcement Learningcited by 119
  12. 2021Interpretable Deep Learning: Interpretation, Interpretability, Trustworthiness, and Beyondcited by 118
12 more
  1. 2020Explainability in Deep Reinforcement Learningcited by 97
  2. 2019Interestingness Elements for Explainable Reinforcement Learning: Understanding Agents' Capabilities and Limitationscited by 50
  3. 2018Establishing Appropriate Trust via Critical Statescited by 48
  4. 2022Deep Surrogate Assisted Generation of Environmentscited by 40
  5. 2017Enabling Robots to Communicate their Objectivescited by 35
  6. 2019Exploring Computational User Models for Agent Policy Summarizationcited by 29
  7. 2022Search-Based Testing of Reinforcement Learningcited by 13
  8. 2021MDPFuzz: Testing Models Solving Markov Decision Processescited by 12
  9. 2018Evolutionary Diversity Optimization Using Multi-Objective Indicatorscited by 9
  10. 2019Benchmarking Surrogate-Assisted Genetic Recommender Systemscited by 4
  11. 2023CROP: Towards Distributional-Shift Robust Reinforcement Learning using Compact Reshaped Observation Processingcited by 3
  12. 2018Inheritance-Based Diversity Measures for Explicit Convergence Control in Evolutionary Algorithmscited by 2

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Interpret Policies in Deep Reinforcement Learning using SILVER with RL-Guided Labeling: A Model-level Approach to High-dimensional and Multi-action Environmentscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025BASIL: Best-Action Symbolic Interpretable Learning for Evolving Compact RL Policies
  2. 2026Code Evolution for Control: Synthesizing Policies via LLM-Driven Evolutionary Search
  3. 2025Building surrogate models using trajectories of agents trained by Reinforcement Learning
  4. 2026Discovering Interpretable Multi-Parameter Control Policies for Evolutionary Algorithms Using Deep Reinforcement Learning
  5. 2025Balancing Interpretability and Performance in Reinforcement Learning: An Adaptive Spectral Based Linear Approach
  6. 2025Efficient On-Policy Reinforcement Learning via Exploration of Sparse Parameter Space
  7. 2026Evolutionary Bilevel Reward Shaping for Generalization in Reinforcement Learning
  8. 2026Synthetic Monitoring Environments for Reinforcement Learning