mlsanityData to 7 Oct 2026

BlendRL: A Framework for Merging Symbolic and Neural Policy Learning

arXiv 2410.11689 ↗First posted 15 Oct 2024cs.LG cs.AIcited by 14

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Humans can leverage both symbolic reasoning and intuitive reactions. In contrast, reinforcement learning policies are typically encoded in either opaque systems like neural networks or symbolic systems that rely on predefined symbols and rules. This disjointed approach severely limits the agents' capabilities, as they often lack either the flexible low-level reaction characteristic of neural agents or the interpretable reasoning of symbolic agents. To overcome this challenge, we introduce BlendRL, a neuro-symbolic RL framework that harmoniously integrates both paradigms within RL agents that use mixtures of both logic and neural policies. We empirically demonstrate that BlendRL agents outperform both neural and symbolic baselines in standard Atari environments, and showcase their robustness to environmental changes. Additionally, we analyze the interaction between neural and symbolic policies, illustrating how their hybrid use helps agents overcome each other's limitations.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 58 of its 91 references that are in the map. First posted on arXiv 15 Oct 2024.

Builds on

58 of 91 references.

Closest ancestors · 2

Same direction, newest first

  1. 2017Proximal Policy Optimization Algorithms20 Jul
  2. 2017Thinking Fast and Slow with Deep Learning and Tree Search23 May

Other references · 56

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2023Segment Anythingcited by 8,158
  3. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  4. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  5. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  6. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  7. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  8. 2020Object-Centric Learning with Slot Attentioncited by 865
  9. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
  10. 2023ViperGPT: Visual Inference via Python Execution for Reasoningcited by 591
  11. 2019The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervisioncited by 369
  12. 2020Agent57: Outperforming the Atari Human Benchmarkcited by 300
38 more
  1. 2023Fast Segment Anythingcited by 286
  2. 2019Stabilizing Transformers for Reinforcement Learningcited by 262
  3. 2017Learning Explanatory Rules from Noisy Datacited by 210
  4. 2018Programmatically Interpretable Reinforcement Learningcited by 192
  5. 2020SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decompositioncited by 190
  6. 2023From Word Models to World Models: Translating from Natural Language to the Probabilistic Language of Thoughtcited by 118
  7. 2018Generalization and Regularization in DQNcited by 117
  8. 2019CrossQ: Batch Normalization in Deep Reinforcement Learning for Greater Sample Efficiency and Simplicitycited by 89
  9. 2018SDRL: Interpretable and Data-efficient Deep Reinforcement Learning Leveraging Symbolic Planningcited by 77
  10. 2021LTL2Action: Generalizing LTL Instructions for Multi-Task RLcited by 72
  11. 2020Combining Reinforcement Learning and Constraint Programming for Combinatorial Optimizationcited by 63
  12. 2023Coupling Large Language Models with Logic Programming for Robust and General Reasoning from Textcited by 59
  13. 2019Neural Logic Reinforcement Learningcited by 53
  14. 2023An Investigation into Pre-Training Object-Centric Representations for Reinforcement Learningcited by 49
  15. 2023Interpretable and Explainable Logical Policies via Neurally Guided Symbolic Abstractioncited by 45
  16. 2020Self-supervised Visual Reinforcement Learning with Object-centric Representationscited by 43
  17. 2020Encoding formulas as deep networks: Reinforcement learning for zero-shot execution of LTL formulascited by 36
  18. 2019An Explicitly Relational Neural Network Architecturecited by 35
  19. 2023Leveraging Large Language Models to Generate Answer Set Programscited by 33
  20. 2021Neuro-Symbolic Reinforcement Learning with First-Order Logiccited by 30
  21. 2023OCAtari: Object-Centric Atari 2600 Reinforcement Learning Environmentscited by 30
  22. 2021Differentiable Inductive Logic Programming for Structured Examplescited by 26
  23. 2021Adaptive Rational Activations to Boost Deep Reinforcement Learningcited by 23
  24. 2021Semantic Tracklets: An Object-Centric Representation for Visual Multi-Agent Reinforcement Learningcited by 18
  25. 2024Pix2Code: Learning to Compose Neural Visual Concepts as Programscited by 18
  26. 2024Interpretable and Editable Programmatic Tree Policies for Reinforcement Learningcited by 18
  27. 2021In a Nutshell, the Human Asked for This: Latent Goals for Following Temporal Specificationscited by 17
  28. 2022Boosting Object Representation Learning via Motion and Object Continuitycited by 17
  29. 2023Learning by Self-Explainingcited by 17
  30. 2024End-to-End Neuro-Symbolic Reinforcement Learning with Textual Explanationscited by 17
  31. 2021Neuro-Symbolic Forward Reasoningcited by 16
  32. 2022GALOIS: Boosting Deep Reinforcement Learning via Generalizable Logic Synthesiscited by 16
  33. 2023Deep Explainable Relational Reinforcement Learning: A Neuro-Symbolic Approachcited by 15
  34. 2023Learning Dynamic Attribute-factored World Models for Efficient Multi-object Reinforcement Learningcited by 15
  35. 2024Towards Truly Zero-shot Compositional Visual Reasoning with LLMs as Programmerscited by 14
  36. 2024Neural Concept Bindercited by 14
  37. 2023Learning Differentiable Logic Programs for Abstract Visual Reasoningcited by 12
  38. 2024HackAtari: Atari Learning Environments for Robust and Continual Reinforcement Learningcited by 11

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 7; 2026: 6.

14 papers.

Sort:

By area:

  1. 2026Incantation: Natural Language as the Action Interface for Multi-Entity Video World Modelscited by 10
  2. 2025Tapas Are Free! Training-Free Adaptation of Programmatic Agents via LLM-Guided Program Synthesis in Dynamic Environmentscited by 9
  3. 2024Interpretable end-to-end Neurosymbolic Reinforcement Learning agentscited by 4
  4. 2025Reinforcement Learning with Physics-Informed Symbolic Program Priors for Zero-Shot Wireless Indoor Navigationcited by 4
  5. 2025Evaluating Interpretable Reinforcement Learning by Distilling Policies into Programscited by 3
  6. 2025Fodor and Pylyshyn's Legacy: Still No Human-like Systematic Compositionality in Neural Networkscited by 3
  7. 2026Learning Explicit Behavioral Models with Adaptive Questions and World-Model Probescited by 2
  8. 2025Inducing, Detecting and Characterising Neural Modules: A Pipeline for Functional Interpretability in Reinforcement Learningcited by 1
  9. 2025Human-Allied Relational Reinforcement Learningcited by 1
  10. 2026REFLEX: Reflective Evolution from LLM Experiencecited by 1
  11. 2025LOOP: A Plug-and-Play Neuro-Symbolic Framework for Enhancing Planning in Autonomous Systemscited by 0
  12. 2026Boosting deep Reinforcement Learning using pretraining with Logical Optionscited by 0
2 more
  1. 2026GRAIL: Autonomous Concept Grounding for Neuro-Symbolic Reinforcement Learningcited by 0
  2. 2026From Black Box to Executable Logic: Explainable Reinforcement Learning through Prolog Expert Systemscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Neuro-Symbolic Computer Use: Learning Reusable Policies for Reliable and Efficient Execution
  2. 2024A Neuro-Symbolic Approach to Multi-Agent RL for Interpretability and Probabilistic Decision Making
  3. 2026Neural-Symbolic Integration with Evolvable Policies
  4. 2024VisualPredicator: Learning Abstract World Models with Neuro-Symbolic Predicates for Robot Planning
  5. 2025Bridging Symbolic Control and Neural Reasoning in LLM Agents -- The Structured Cognitive Loop
  6. 2022Neuro-symbolic Natural Logic with Introspective Revision for Natural Language Inference
  7. 2026Breaking Task Impasses Quickly: Adaptive Neuro-Symbolic Learning for Open-World Robotics
  8. 2026SEARL: Joint Optimization of Policy and Tool Graph Memory for Self-Evolving Agents