mlsanityData to 7 Oct 2026

GRAIL: Autonomous Concept Grounding for Neuro-Symbolic Reinforcement Learning

arXiv 2604.16871 ↗First posted 18 Apr 2026cs.AI cs.LGcited by 0

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Neuro-symbolic Reinforcement Learning (NeSy-RL) combines symbolic reasoning with gradient-based optimization to achieve interpretable and generalizable policies. Relational concepts, such as "left of" or "close by", serve as foundational building blocks that structure how agents perceive and act. However, conventional approaches require human experts to manually define these concepts, limiting adaptability since concept semantics vary across environments. We propose GRAIL (Grounding Relational Agents through Interactive Learning), a framework that autonomously grounds relational concepts through environmental interaction. GRAIL leverages large language models (LLMs) to provide generic concept representations as weak supervision, then refines them to capture environment-specific semantics. This approach addresses both sparse reward signals and concept misalignment prevalent in underdetermined environments. Experiments on the Atari games Kangaroo, Seaquest, and Skiing demonstrate that GRAIL matches or outperforms agents with manually crafted concepts in simplified settings, and reveals informative trade-offs between reward maximization and high-level goal completion in the full environment.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 62 of its 81 references that are in the map. First posted on arXiv 18 Apr 2026.

Builds on

62 of 81 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024BlendRL: A Framework for Merging Symbolic and Neural Policy Learning15 Oct
  2. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 60

  1. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  3. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  4. 2020Concept Bottleneck Modelscited by 1,022
  5. 2020Object-Centric Learning with Slot Attentioncited by 865
  6. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
  7. 2019The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervisioncited by 369
  8. 2020Agent57: Outperforming the Atari Human Benchmarkcited by 300
  9. 2023Fast Segment Anythingcited by 286
  10. 2021MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learningcited by 281
  11. 2019Stabilizing Transformers for Reinforcement Learningcited by 262
  12. 2021Conditional Object-Centric Learning from Videocited by 247
38 more
  1. 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 220
  2. 2017Learning Explanatory Rules from Noisy Datacited by 210
  3. 2018Programmatically Interpretable Reinforcement Learningcited by 192
  4. 2020SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decompositioncited by 190
  5. 2020Right for the Right Concept: Revising Neuro-Symbolic Concepts by Interacting with their Explanationscited by 94
  6. 2019CrossQ: Batch Normalization in Deep Reinforcement Learning for Greater Sample Efficiency and Simplicitycited by 89
  7. 2018SDRL: Interpretable and Data-efficient Deep Reinforcement Learning Leveraging Symbolic Planningcited by 77
  8. 2022Deep Differentiable Logic Gate Networkscited by 67
  9. 2020Combining Reinforcement Learning and Constraint Programming for Combinatorial Optimizationcited by 63
  10. 2019Neural Logic Reinforcement Learningcited by 53
  11. 2022Predicate Invention for Bilevel Planningcited by 49
  12. 2023An Investigation into Pre-Training Object-Centric Representations for Reinforcement Learningcited by 49
  13. 2023Interpretable and Explainable Logical Policies via Neurally Guided Symbolic Abstractioncited by 45
  14. 2020Self-supervised Visual Reinforcement Learning with Object-centric Representationscited by 43
  15. 2023Explainable Deep Reinforcement Learning: State of the Art and Challengescited by 33
  16. 2023What's Left? Concept Grounding with Logic-Enhanced Foundation Modelscited by 33
  17. 2021Neuro-Symbolic Reinforcement Learning with First-Order Logiccited by 30
  18. 2023OCAtari: Object-Centric Atari 2600 Reinforcement Learning Environmentscited by 30
  19. 2024Entity-Centric Reinforcement Learning for Object Manipulation from Pixelscited by 27
  20. 2021Differentiable Inductive Logic Programming for Structured Examplescited by 26
  21. 2021Interactive Disentanglement: Learning Concepts by Interacting with their Prototype Representationscited by 24
  22. 2023Neurosymbolic Reinforcement Learning and Planning: A Surveycited by 24
  23. 2024SOLD: Slot Object-Centric Latent Dynamics Models for Relational Manipulation Learning from Pixelscited by 23
  24. 2024Human-in-the-loop or AI-in-the-loop? Automate or Collaborate?cited by 23
  25. 2024MetaUrban: An Embodied AI Simulation Platform for Urban Micromobilitycited by 19
  26. 2021Semantic Tracklets: An Object-Centric Representation for Visual Multi-Agent Reinforcement Learningcited by 18
  27. 2024Pix2Code: Learning to Compose Neural Visual Concepts as Programscited by 18
  28. 2024Interpretable and Editable Programmatic Tree Policies for Reinforcement Learningcited by 18
  29. 2022Boosting Object Representation Learning via Motion and Object Continuitycited by 17
  30. 2024End-to-End Neuro-Symbolic Reinforcement Learning with Textual Explanationscited by 17
  31. 2022GALOIS: Boosting Deep Reinforcement Learning via Generalizable Logic Synthesiscited by 16
  32. 2023Deep Explainable Relational Reinforcement Learning: A Neuro-Symbolic Approachcited by 15
  33. 2024Neural Concept Bindercited by 14
  34. 2023Learning Differentiable Logic Programs for Abstract Visual Reasoningcited by 12
  35. 2024HackAtari: Atari Learning Environments for Robust and Continual Reinforcement Learningcited by 11
  36. 2025Learning Interactive World Model for Object-Centric Reinforcement Learningcited by 11
  37. 2025Building Intelligent Agents with Neuro-Symbolic Conceptscited by 10
  38. 2025Object Centric Concept Bottleneckscited by 10

50 most cited of 60 All 60 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Ground-Compose-Reinforce: Grounding Language in Agentic Behaviours using Limited Data
  2. 2026Lifting Traces to Logic: Programmatic Skill Induction with Neuro-Symbolic Learning for Long-Horizon Agentic Tasks
  3. 2022Learning Rules from Rewards
  4. 2026Reward-Zero: Language Embedding Driven Implicit Reward Mechanisms for Reinforcement Learning
  5. 2025ARM-FM: Automated Reward Machines via Foundation Models for Compositional Reinforcement Learning
  6. 2026From Passive Reuse to Active Reasoning: Grounding Large Language Models for Neuro-Symbolic Experience Replay
  7. 2026Skill0.5: Joint Skill Internalization and Utilization for Out-of-Distribution Generalization in Agentic Reinforcement Learning
  8. 2026Neurosymbolic Reasoning with Incremental Knowledge for Sample Efficient Hierarchical Reinforcement Learning