mlsanityData to 7 Oct 2026

Mitigating Information Loss in Tree-Based Reinforcement Learning via Direct Optimization

arXiv 2408.08761 ↗First posted 16 Aug 2024cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

Reinforcement learning (RL) has seen significant success across various domains, but its adoption is often limited by the black-box nature of neural network policies, making them difficult to interpret. In contrast, symbolic policies allow representing decision-making strategies in a compact and interpretable way. However, learning symbolic policies directly within on-policy methods remains challenging. In this paper, we introduce SYMPOL, a novel method for SYMbolic tree-based on-POLicy RL. SYMPOL employs a tree-based model integrated with a policy gradient method, enabling the agent to learn and adapt its actions while maintaining a high level of interpretability. We evaluate SYMPOL on a set of benchmark RL tasks, demonstrating its superiority over alternative tree-based RL approaches in terms of performance and interpretability. Unlike existing methods, it enables gradient-based, end-to-end learning of interpretable, axis-aligned decision trees within standard on-policy RL algorithms. Therefore, SYMPOL can become the foundation for a new class of interpretable RL based on decision trees. Our implementation is available under: https://github.com/s-marton/sympol

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 39 of its 59 references that are in the map. First posted on arXiv 16 Aug 2024.

Builds on

39 of 59 references.

References · 39

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,455
  4. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  5. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  6. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,194
  7. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 750
  8. 2023Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Taskscited by 320
  9. 2019Neural Oblivious Decision Ensembles for Deep Learning on Tabular Datacited by 200
  10. 2018Programmatically Interpretable Reinforcement Learningcited by 192
  11. 2018Verifiable Reinforcement Learning via Policy Extractioncited by 176
  12. 2019Sparse Sequence-to-Sequence Modelscited by 167
27 more
  1. 2018Generalization and Regularization in DQNcited by 117
  2. 2021NODE-GAM: Neural Generalized Additive Model for Interpretable Deep Learningcited by 88
  3. 2018Toward Interpretable Deep Reinforcement Learning with Linear Model U-Treescited by 65
  4. 2021Learning to Synthesize Programs as Interpretable and Generalizable Policiescited by 59
  5. 2022EnvPool: A Highly Parallel Reinforcement Learning Environment Execution Enginecited by 58
  6. 2019Neural Logic Reinforcement Learningcited by 53
  7. 2023Interpretable and Explainable Logical Policies via Neurally Guided Symbolic Abstractioncited by 45
  8. 2021Iterative Bounding MDPs: Learning Interpretable Policies via Non-Interpretable Methodscited by 31
  9. 2019Conservative Q-Improvement: Reinforcement Learning for an Interpretable Decision-Tree Policycited by 29
  10. 2024Interpretable Concept Bottlenecks to Align Reinforcement Learning Agentscited by 23
  11. 2024Interpretable and Editable Programmatic Tree Policies for Reinforcement Learningcited by 18
  12. 2024End-to-End Neuro-Symbolic Reinforcement Learning with Textual Explanationscited by 17
  13. 2022GALOIS: Boosting Deep Reinforcement Learning via Generalizable Logic Synthesiscited by 16
  14. 2020Evolutionary learning of interpretable decision treescited by 14
  15. 2022MAVIPER: Learning Decision Tree Policies for Interpretable Multi-Agent Reinforcement Learningcited by 12
  16. 2023GradTree: Learning Axis-Aligned Decision Trees with Gradient Descentcited by 11
  17. 2024HackAtari: Atari Learning Environments for Robust and Continual Reinforcement Learningcited by 11
  18. 2022MIXRTs: Toward Interpretable Multi-Agent Reinforcement Learning via Mixing Recurrent Soft Decision Treescited by 10
  19. 2023GRANDE: Gradient-Based Decision Tree Ensembles for Tabular Datacited by 10
  20. 2023Efficient Symbolic Policy Learning with Differentiable Symbolic Expressioncited by 9
  21. 2023Optimal Decision Tree Policies for Markov Decision Processescited by 8
  22. 2021Neural-to-Tree Policy Distillation with Policy Improvement Criterioncited by 5
  23. 2021Natural Language Specification of Reinforcement Learning Policies through Differentiable Decision Treescited by 4
  24. 2024LICORICE: Label-Efficient Concept-Based Interpretable Reinforcement Learningcited by 4
  25. 2019Learning Programmatically Structured Representations with Perceptor Gradientscited by 3
  26. 2022Q-learning with online random forestscited by 2
  27. 2024Gradient Boosting Reinforcement Learningcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 2.

5 papers.

  1. 2025Evaluating Interpretable Reinforcement Learning by Distilling Policies into Programscited by 3
  2. 2025Information Templates: A New Paradigm for Intelligent Active Feature Acquisitioncited by 2
  3. 2025Inducing, Detecting and Characterising Neural Modules: A Pipeline for Functional Interpretability in Reinforcement Learningcited by 1
  4. 2026Approximation-Free Differentiable Oblique Decision Treescited by 0
  5. 2026ConceptTree: Bringing Semantic Transparency to Black-Box Decision Making for Robotic Manipulationcited by 0