mlsanityData to 14 Oct 2026

VisualPredicator: Learning Abstract World Models with Neuro-Symbolic Predicates for Robot Planning

arXiv 2410.23156 ↗First posted 30 Oct 2024cs.AI cs.CV cs.LGcited by 39

On the map: World models → Embodied and robot world models

Abstract (arXiv)

Broadly intelligent agents should form task-specific abstractions that selectively expose the essential elements of a task, while abstracting away the complexity of the raw sensorimotor space. In this work, we present Neuro-Symbolic Predicates, a first-order abstraction language that combines the strengths of symbolic and neural knowledge representations. We outline an online algorithm for inventing such predicates and learning abstract world models. We compare our approach to hierarchical reinforcement learning, vision-language model planning, and symbolic predicate invention approaches, on both in- and out-of-distribution tasks across five simulated robotic domains. Results show that our approach offers better sample complexity, stronger out-of-distribution generalization, and improved interpretability.

Key facts. Cited by 39 papers in the map (data through 14 Oct 2026). Placed in Embodied and robot world models, one of 11 directions in World models. Builds on 25 of its 43 references that are in the map. First posted on arXiv 30 Oct 2024.

Builds on

25 of 43 references.

References · 25

  1. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  2. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  3. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,210
  4. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,081
  5. 2023ViperGPT: Visual Inference via Python Execution for Reasoningcited by 603
  6. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 571
  7. 2022Visual Programming: Compositional visual reasoning without trainingcited by 516
  8. 2024ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulationcited by 315
  9. 2020Integrated Task and Motion Planningcited by 290
  10. 2023Look Before You Leap: Unveiling the Power of GPT-4V in Robotic Vision-Language Planningcited by 161
  11. 2024WorldCoder, a Model-Based LLM Agent: Building World Models by Writing Code and Interacting with the Environmentcited by 107
  12. 2023Generative Skill Chaining: Long-Horizon Skill Planning with Diffusion Modelscited by 91
13 more
  1. 2021Augmenting Reinforcement Learning with Behavior Primitives for Diverse Manipulation Taskscited by 78
  2. 2015Reinforcement Learning with Parameterized Actionscited by 63
  3. 2021Learning Symbolic Operators for Task and Motion Planningcited by 58
  4. 2022Learning Neuro-Symbolic Skills for Bilevel Planningcited by 58
  5. 2022Predicate Invention for Bilevel Planningcited by 49
  6. 2021Learning Neuro-Symbolic Relational Transition Models for Bilevel Planningcited by 43
  7. 2021Long-Horizon Manipulation of Unknown Objects via Task and Motion Planning with Estimated Affordancescited by 40
  8. 2024InterPreT: Interactive Predicate Learning from Language Feedback for Generalizable Task Planningcited by 31
  9. 2022Learning Efficient Abstract Planning Models that Choose What to Predictcited by 17
  10. 2024Partially Observable Task and Motion Planning with Uncertainty and Risk Awarenesscited by 17
  11. 2023Rapid Motor Adaptation for Robotic Manipulator Armscited by 14
  12. 2019Learning Portable Representations for High-Level Planningcited by 11
  13. 2019Blocks: Collaborative and Persistent Augmented Reality Experiencescited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 15; 2026: 20.

39 papers.

Sort:

By area:

  1. 2024Combining Induction and Transduction for Abstract Reasoningcited by 53
  2. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 50
  3. 2026World Model for Robot Learning: A Comprehensive Surveycited by 46
  4. 2025PoE-World: Compositional World Modeling with Products of Programmatic Expertscited by 26
  5. 2025LLM-Guided Probabilistic Program Induction for POMDP Model Estimationcited by 12
  6. 2024From Pixels to Predicates: Learning Symbolic World Models via Pretrained Vision-Language Modelscited by 11
  7. 2025DMWM: Dual-Mind World Model with Long-Term Imaginationcited by 10
  8. 2025Four Principles for Physically Interpretable World Modelscited by 10
  9. 2024Open-World Task and Motion Planning via Vision-Language Model Generated Constraintscited by 9
  10. 2025ExoPredicator: Learning Abstract Models of Dynamic Worlds for Robot Planningcited by 9
  11. 2025Language Models For Generalised PDDL Planning: Synthesising Sound and Programmatic Policiescited by 7
  12. 2026H-WM: Robotic Task and Motion Planning Guided by Hierarchical World Modelcited by 7
27 more
  1. 2026Neuro-Symbolic Synergy for World Modelingcited by 6
  2. 2025ViPlan: A Benchmark for Visual Planning with Symbolic Predicates and Vision-Language Modelscited by 5
  3. 2026From World Models to World Action Models: A Concise Tutorial for Roboticscited by 4
  4. 2024Physically Interpretable World Models via Weakly Supervised Representation Learningcited by 3
  5. 2025Vision Language Models Cannot Plan, but Can They Formalize?cited by 3
  6. 2025SLAP: Shortcut Learning for Abstract Planningcited by 3
  7. 2025VDAWorld: World Modelling via VLM-Directed Abstraction and Simulationcited by 3
  8. 2026Learning Abstractions for Hierarchical Planning in Program-Synthesis Agentscited by 3
  9. 2026Learning Lifted Action Models from Unsupervised Visual Tracescited by 3
  10. 2026Kintsugi: Learning Policies by Repairing Executable Knowledge Basescited by 3
  11. 2025DR. WELL: Dynamic Reasoning and Learning with Symbolic World Model for Embodied LLM-Based Multi-Agent Collaborationcited by 2
  12. 2025Unifying Deep Predicate Invention with Pre-trained Foundation Modelscited by 2
  13. 2025Learning Hierarchical Domain Models Through Environment-Grounded Interactioncited by 1
  14. 2025Coloring Between the Lines: Personalization in the Null Space of Planning Constraintscited by 1
  15. 2025PSALM-V: Automating Symbolic Planning in Interactive Visual Environments with Large Language Modelscited by 1
  16. 2026Joint Learning of Hierarchical Neural Options and Abstract World Modelcited by 1
  17. 2026StageGuard: Learning Stage Transitions for Long-Horizon Robot Tasks via Agentic Distillationcited by 1
  18. 2026EMPIRIC: Experiment-Driven Learning of Residual World Models for Robot Planningcited by 1
  19. 2026CABTO: Context-Aware Behavior Tree Grounding for Robot Manipulationcited by 0
  20. 2026Cross-Domain Demo-to-Code via Neurosymbolic Counterfactual Reasoningcited by 0
  21. 2026Regression Test Selection for Updated Capability Modules in Compositional ML Systems via Atomic-Quality Probescited by 0
  22. 2026Learning Bilevel Policies over Symbolic World Models for Long-Horizon Planningcited by 0
  23. 2026Legislating World-Model-Based Planning with Legal Reasoningcited by 0
  24. 2026Bridging Learned Visual Perception and Symbolic Belief-Space Planningcited by 0
  25. 2026TANDEM: Task and Motion Planning with As-Needed Demonstrations for Efficient Vision-Language-Action Model Fine-tuningcited by 0
  26. 2026BeliefGraph-JEPA: Structured Latent World Models for Action-Conditioned Time Seriescited by 0
  27. 2026Who to Trust in Multi-Robot Systems: Joint Robot and Object Hypothesis Trust Inference via Neural-Symbolic Cross-Validationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Build on Priors: Vision--Language--Guided Neuro-Symbolic Imitation Learning for Data-Efficient Real-World Robot Manipulation
  2. 2026Learning Structured Robot Policies from Vision-Language Models via Synthetic Neuro-Symbolic Supervision
  3. 2024From Real World to Logic and Back: Learning Generalizable Relational Concepts For Long Horizon Robot Planning
  4. 2025A Neuro-Symbolic Framework for Reasoning under Perceptual Uncertainty: Bridging Continuous Perception and Discrete Symbolic Planning
  5. 2026Flow Policies as Actions of Skill-Level World Models: Learned and Symbolic Abstractions for Long-Horizon Planning
  6. 2026Emergent Neural Automaton Policies: Learning Symbolic Structure from Visuomotor Trajectories ↗
  7. 2026Breaking Task Impasses Quickly: Adaptive Neuro-Symbolic Learning for Open-World Robotics
  8. 2026STRIPS-WM: Learning Grounded Propositional STRIPS-style World Models from Images