mlsanityData to 7 Oct 2026

Building Open-Ended Embodied Agent via Language-Policy Bidirectional Adaptation

arXiv 2401.00006 ↗First posted 12 Dec 2023cs.AIcited by 6

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

Building embodied agents on integrating Large Language Models (LLMs) and Reinforcement Learning (RL) have revolutionized human-AI interaction: researchers can now leverage language instructions to plan decision-making for open-ended tasks. However, existing research faces challenges in meeting the requirement of open-endedness. They typically either train LLM/RL models to adapt to a fixed counterpart, limiting exploration of novel skills and hindering the efficacy of human-AI interaction. To this end, we present OpenPAL, a co-training framework comprising two stages: (1) fine-tuning a pre-trained LLM to translate human instructions into goals for planning, and goal-conditioned training a policy for decision-making; (2) co-training to align the LLM and policy, achieving instruction open-endedness. We conducted experiments using Contra, an open-ended FPS game, demonstrating that an agent trained with OpenPAL not only comprehends arbitrary instructions but also exhibits efficient execution. These results suggest that OpenPAL holds the potential to construct open-ended embodied agents in practical scenarios.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 43 of its 65 references that are in the map. First posted on arXiv 12 Dec 2023.

Builds on

43 of 65 references.

References · 43

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2017Proximal Policy Optimization Algorithmscited by 14,441
  5. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  6. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  7. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  8. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  9. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  10. 2017Hindsight Experience Replaycited by 1,096
  11. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  12. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,026
31 more
  1. 2018Exploration by Random Network Distillationcited by 966
  2. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  3. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 733
  4. 2023Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agentscited by 396
  5. 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 350
  6. 2019On the Utility of Learning about Humans for Human-AI Coordinationcited by 345
  7. 2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 328
  8. 2017Overcoming Exploration in Reinforcement Learning with Demonstrationscited by 322
  9. 2018Human-level performance in first-person multiplayer games with population-based deep reinforcement learningcited by 260
  10. 2020Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Designcited by 260
  11. 2020First return, then explorecited by 256
  12. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 252
  13. 2020Language Conditioned Imitation Learning over Unstructured Datacited by 208
  14. 2021Simple but Effective: CLIP Embeddings for Embodied AIcited by 191
  15. 2023Guiding Pretraining in Reinforcement Learning with Large Language Modelscited by 185
  16. 2019Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutionscited by 182
  17. 2021Open-Ended Learning Leads to Generally Capable Agentscited by 177
  18. 2021Discovering and Achieving Goals via World Modelscited by 131
  19. 2018Unsupervised Control Through Non-Parametric Discriminative Rewardscited by 102
  20. 2017Grounded Language Learning in a Simulated 3D Worldcited by 100
  21. 2020Maximum Entropy Gain Exploration for Long Horizon Multi-goal Reinforcement Learningcited by 100
  22. 2019Open-ended Learning in Symmetric Zero-sum Gamescited by 96
  23. 2020Learning with AMIGo: Adversarially Motivated Intrinsic Goalscited by 86
  24. 2019Hierarchical Foresight: Self-Supervised Learning of Long-Horizon Tasks via Visual Subgoal Generationcited by 83
  25. 2019Keeping Your Distance: Solving Sparse Reward Tasks Using Self-Balancing Shaped Rewardscited by 76
  26. 2019Exploration via Hindsight Goal Generationcited by 53
  27. 2023Anti-Exploration by Random Network Distillationcited by 49
  28. 2023Enabling Intelligent Interactions between an Agent and an LLM: A Reinforcement Learning Approachcited by 28
  29. 2019On the Utility of Model Learning in HRIcited by 21
  30. 2020Goal-Conditioned End-to-End Visuomotor Control for Versatile Skill Primitivescited by 7
  31. 2023Magnetic Field-Based Reward Shaping for Goal-Conditioned Reinforcement Learningcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 3; 2026: 1.

6 papers.

  1. 2025A Vision-Language-Action-Critic Model for Robotic Real-World Reinforcement Learningcited by 66
  2. 2024A Survey on Game Playing Agents and Large Models: Methods, Applications, and Challengescited by 22
  3. 2025Parallels Between VLA Model Post-Training and Human Motor Learning: Progress, Challenges, and Trendscited by 17
  4. 2025Neural Brain: A Neuroscience-inspired Framework for Embodied Agentscited by 10
  5. 2024A Survey On Enhancing Reinforcement Learning in Complex Environments: Insights from Human and LLM Feedbackcited by 9
  6. 2026HELP: Human-Efficient Large-Scale Robot Post-Training with Rollout Segmentationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Goal Inference from Open-Ended Dialog
  2. 2024OPEx: A Component-Wise Analysis of LLM-Centric Agents in Embodied Instruction Following
  3. 2023Collaborating with language models for embodied reasoning
  4. 2023Plan, Eliminate, and Track -- Language Models are Good Teachers for Embodied Agents
  5. 2025Grounding Multimodal LLMs to Embodied Agents that Ask for Help with Reinforcement Learning
  6. 2025HELP: Hierarchical Embodied Language Planner for Household Tasks
  7. 2024From Multimodal LLMs to Generalist Embodied Agents: Methods and Lessons
  8. 2024Do We Really Need a Complex Agent System? Distill Embodied Agent into a Single Model