mlsanityData to 14 Oct 2026

Efficient Symbolic Policy Learning with Differentiable Symbolic Expression

arXiv 2311.02104 ↗First posted 2 Nov 2023cs.LG cs.AIcited by 10

Not on the map. Why?

Abstract (arXiv)

Deep reinforcement learning (DRL) has led to a wide range of advances in sequential decision-making tasks. However, the complexity of neural network policies makes it difficult to understand and deploy with limited computational resources. Currently, employing compact symbolic expressions as symbolic policies is a promising strategy to obtain simple and interpretable policies. Previous symbolic policy methods usually involve complex training processes and pre-trained neural network policies, which are inefficient and limit the application of symbolic policies. In this paper, we propose an efficient gradient-based learning method named Efficient Symbolic Policy Learning (ESPL) that learns the symbolic policy from scratch in an end-to-end way. We introduce a symbolic network as the search space and employ a path selector to find the compact symbolic policy. By doing so we represent the policy with a differentiable symbolic expression and train it in an off-policy manner which further improves the efficiency. In addition, in contrast with previous symbolic policies which only work in single-task RL because of complexity, we expand ESPL on meta-RL to generate symbolic policies for unseen tasks. Experimentally, we show that our approach generates symbolic policies with higher performance and greatly improves data efficiency for single-task RL. In meta-RL, we demonstrate that compared with neural network policies the proposed symbolic policy achieves higher performance and efficiency and shows the potential to be interpretable.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 44 of its 61 references that are in the map. First posted on arXiv 2 Nov 2023.

Builds on

44 of 61 references.

References · 44

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2016Densely Connected Convolutional Networkscited by 6,720
  4. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  5. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  6. 2015Continuous control with deep reinforcement learningcited by 3,407
  7. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,739
  8. 2015Trust Region Policy Optimizationcited by 2,580
  9. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  10. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  11. 2019Searching for MobileNetV3cited by 1,994
  12. 2018DARTS: Differentiable Architecture Searchcited by 1,798
32 more
  1. 2016Neural Architecture Search with Reinforcement Learningcited by 1,499
  2. 2015End-to-End Training of Deep Visuomotor Policiescited by 875
  3. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 673
  4. 2017Learning Sparse Neural Networks through L0 Regularizationcited by 583
  5. 2019AI Feynman: a Physics-Inspired Method for Symbolic Regressioncited by 548
  6. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 503
  7. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 389
  8. 2017Meta-SGD: Learning to Learn Quickly for Few-Shot Learningcited by 379
  9. 2020Discovering Symbolic Models from Deep Learning with Inductive Biasescited by 344
  10. 2019Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradientscited by 309
  11. 2019Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentationcited by 260
  12. 2019VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learningcited by 217
  13. 2018Programmatically Interpretable Reinforcement Learningcited by 193
  14. 2018Verifiable Reinforcement Learning via Policy Extractioncited by 177
  15. 2018Learning Equations for Extrapolation and Controlcited by 156
  16. 2017Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximationcited by 121
  17. 2016Extrapolation and learning equationscited by 110
  18. 2018ProMP: Proximal Meta-Policy Searchcited by 91
  19. 2016Training Sparse Neural Networkscited by 73
  20. 2019Meta-Q-Learningcited by 62
  21. 2021Learning to Synthesize Programs as Interpretable and Generalizable Policiescited by 59
  22. 2017Interpretable Policies for Reinforcement Learning by Genetic Programmingcited by 48
  23. 2021AdaRL: What, Where, and How to Adapt in Transfer Reinforcement Learningcited by 48
  24. 2021Effective Sparsification of Neural Networks with Global Sparsity Constraintcited by 45
  25. 2021Recomposing the Reinforcement Learning Building Blocks with Hypernetworkscited by 35
  26. 2016Programming with a Differentiable Forth Interpretercited by 33
  27. 2018Learning concise representations for regression by evolving networks of treescited by 33
  28. 2019Few-Shot Bayesian Imitation Learning with Logical Program Policiescited by 30
  29. 2020Towards Effective Context for Meta-Reinforcement Learning: an Approach based on Contrastive Learningcited by 29
  30. 2017Learning to Learn: Meta-Critic Networks for Sample Efficient Learningcited by 28
  31. 2020MetaCURE: Meta Reinforcement Learning with Empowerment-Driven Explorationcited by 25
  32. 2016TerpreT: A Probabilistic Programming Language for Program Inductioncited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 2; 2026: 4.

10 papers.

  1. 2025A Comprehensive Survey on Self-Interpretable Neural Networkscited by 18
  2. 2024Mitigating Information Loss in Tree-Based Reinforcement Learning via Direct Optimizationcited by 5
  3. 2024KnowPC: Knowledge-Driven Programmatic Reinforcement Learning for Zero-shot Coordinationcited by 2
  4. 2025Sparse Interpretable Deep Learning with LIES Networks for Symbolic Regressioncited by 2
  5. 2024Towards a Research Community in Interpretable Reinforcement Learning: the InterpPol Workshopcited by 0
  6. 2024Discovering Continuous-Time Memory-Based Symbolic Policies using Genetic Programmingcited by 0
  7. 2026Boosting deep Reinforcement Learning using pretraining with Logical Optionscited by 0
  8. 2026From Passive Reuse to Active Reasoning: Grounding Large Language Models for Neuro-Symbolic Experience Replaycited by 0
  9. 2026SMILE: Bridging Continuous Optimization and Discrete Symbolic Recoverycited by 0
  10. 2026SD-DPC: Sparse Dictionary Differentiable Predictive Controlcited by 0