mlsanityData to 7 Oct 2026

OCALM: Object-Centric Assessment with Language Models

arXiv 2406.16748 ↗First posted 24 Jun 2024cs.LG cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Properly defining a reward signal to efficiently train a reinforcement learning (RL) agent is a challenging task. Designing balanced objective functions from which a desired behavior can emerge requires expert knowledge, especially for complex environments. Learning rewards from human feedback or using large language models (LLMs) to directly provide rewards are promising alternatives, allowing non-experts to specify goals for the agent. However, black-box reward models make it difficult to debug the reward. In this work, we propose Object-Centric Assessment with Language Models (OCALM) to derive inherently interpretable reward functions for RL agents from natural language task descriptions. OCALM uses the extensive world-knowledge of LLMs while leveraging the object-centric nature common to many environments to derive reward functions focused on relational concepts, providing RL agents with the ability to derive policies from task descriptions.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 31 of its 46 references that are in the map. First posted on arXiv 24 Jun 2024.

Builds on

31 of 46 references.

References · 31

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  5. 2017Deep reinforcement learning from human preferencescited by 3,846
  6. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  7. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  8. 2016Concrete Problems in AI Safetycited by 1,582
  9. 2017Hindsight Experience Replaycited by 1,096
  10. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 542
  11. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progresscited by 294
  12. 2017Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agentscited by 267
19 more
  1. 2024Reinforcement learningcited by 236
  2. 2020SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decompositioncited by 190
  3. 2023Text2Reward: Reward Shaping with Language Models for Reinforcement Learningcited by 153
  4. 2021Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotationcited by 117
  5. 2021Reward (Mis)design for Autonomous Drivingcited by 102
  6. 2023Read and Reap the Rewards: Learning to Play Atari with the Help of Instruction Manualscited by 45
  7. 2023Interpretable and Explainable Logical Policies via Neurally Guided Symbolic Abstractioncited by 45
  8. 2023Learning to Intervene on Concept Bottleneckscited by 29
  9. 2021Synthetic Returns for Long-Term Credit Assignmentcited by 27
  10. 2021Differentiable Inductive Logic Programming for Structured Examplescited by 26
  11. 2021Explainable Reinforcement Learning for Broad-XAI: A Conceptual Framework and Surveycited by 24
  12. 2024Interpretable Concept Bottlenecks to Align Reinforcement Learning Agentscited by 23
  13. 2020Inverse Reinforcement Learning with Natural Language Goalscited by 20
  14. 2024Pix2Code: Learning to Compose Neural Visual Concepts as Programscited by 18
  15. 2024Interpretable and Editable Programmatic Tree Policies for Reinforcement Learningcited by 18
  16. 2022Boosting Object Representation Learning via Motion and Object Continuitycited by 17
  17. 2023Natural Language-conditioned Reinforcement Learning with Inside-out Task Language Development and Translationcited by 7
  18. 2024EXPIL: Explanatory Predicate Invention for Learning in Gamescited by 5
  19. 2024Tell me why: Training preferences-based RL with human preferences and step-level explanationscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Reward Design with Language Models
  2. 2024Words as Beacons: Guiding RL Agents with High-Level Language Prompts
  3. 2025ARM-FM: Automated Reward Machines via Foundation Models for Compositional Reinforcement Learning
  4. 2024RL-VLM-F: Reinforcement Learning from Vision Language Foundation Model Feedback
  5. 2024Online Intrinsic Rewards for Decision Making Agents from Large Language Model Feedback
  6. 2026The End of Reward Engineering: How LLMs Are Redefining Multi-Agent Coordination
  7. 2024RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMs
  8. 2026RLAR: An Agentic Reward System for Multi-task Reinforcement Learning on Large Language Models