mlsanityData to 7 Oct 2026

VLGOR: Visual-Language Knowledge Guided Offline Reinforcement Learning for Generalizable Agents

arXiv 2603.22892 ↗First posted 24 Mar 2026cs.LGcited by 1

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

also in 1 more
Abstract (arXiv)

Combining Large Language Models (LLMs) with Reinforcement Learning (RL) enables agents to interpret language instructions more effectively for task execution. However, LLMs typically lack direct perception of the physical environment, which limits their understanding of environmental dynamics and their ability to generalize to unseen tasks. To address this limitation, we propose Visual-Language Knowledge-Guided Offline Reinforcement Learning (VLGOR), a framework that integrates visual and language knowledge to generate imaginary rollouts, thereby enriching the interaction data. The core premise of VLGOR is to fine-tune a vision-language model to predict future states and actions conditioned on an initial visual observation and high-level instructions, ensuring that the generated rollouts remain temporally coherent and spatially plausible. Furthermore, we employ counterfactual prompts to produce more diverse rollouts for offline RL training, enabling the agent to acquire knowledge that facilitates following language instructions while grounding in environments based on visual cues. Experiments on robotic manipulation benchmarks demonstrate that VLGOR significantly improves performance on unseen tasks requiring novel optimal policies, achieving a success rate over 24% higher than the baseline methods.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 28 of its 42 references that are in the map. First posted on arXiv 24 Mar 2026.

Builds on

28 of 42 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Knowledgeable Agents by Offline Reinforcement Learning from Large Language Model Rollouts14 Apr

Other references · 27

  1. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  2. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  3. 2023A Survey on Evaluation of Large Language Modelscited by 1,227
  4. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  5. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 882
  6. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
  7. 2021COMBO: Conservative Offline Model-Based Policy Optimizationcited by 365
  8. 2023Grounding Large Language Models in Interactive Environments with Online Reinforcement Learningcited by 193
  9. 2023Text2Reward: Reward Shaping with Language Models for Reinforcement Learningcited by 153
  10. 2019Language as an Abstraction for Hierarchical Deep Reinforcement Learningcited by 138
  11. 2023Large Language Models as Generalizable Policies for Embodied Taskscited by 86
  12. 2021d3rlpy: An Offline Deep Reinforcement Learning Librarycited by 84
15 more
  1. 2024Driving Everywhere with Large Language Model Policy Adaptationcited by 53
  2. 2023What is Essential for Unseen Goal Generalization of Offline Goal-conditioned RL?cited by 36
  3. 2024True Knowledge Comes from Practice: Aligning LLMs with Embodied Environments via Reinforcement Learningcited by 36
  4. 2023Policy Regularization with Dataset Constraint for Offline Reinforcement Learningcited by 33
  5. 2024Exploring the Robustness of Decision-Level Through Adversarial Attacks on LLM-Based Embodied Modelscited by 31
  6. 2025CAST: Counterfactual Labels Improve Instruction Following in Vision-Language-Action Modelscited by 28
  7. 2023The Generalization Gap in Offline Reinforcement Learningcited by 25
  8. 2024In-Context Decision Transformer: Reinforcement Learning via Hierarchical Chain-of-Thoughtcited by 25
  9. 2023Beyond Uniform Sampling: Offline Reinforcement Learning with Imbalanced Datasetscited by 24
  10. 2023Context Shift Reduction for Offline Meta-Reinforcement Learningcited by 17
  11. 2024Meta-DT: Offline Meta-RL as Conditional Sequence Modeling with World Model Disentanglementcited by 16
  12. 2024Offline Multitask Representation Learning for Reinforcement Learningcited by 14
  13. 2025iManip: Skill-Incremental Learning for Robotic Manipulationcited by 12
  14. 2023Offline Meta Reinforcement Learning with In-Distribution Online Adaptationcited by 10
  15. 2025Mixture-of-Experts Meets In-Context Reinforcement Learningcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Prompt-Driven Exploration: Language as an Exploration Space for VLA Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Policy Learning from Large Vision-Language Model Feedback without Reward Modeling
  2. 2023Towards A Unified Agent with Foundation Models
  3. 2025Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Survey
  4. 2025VLM Q-Learning: Aligning Vision-Language Models for Interactive Decision-Making
  5. 2025GoalLadder: Incremental Goal Discovery with Vision-Language Models
  6. 2026DreamPlan: Efficient Reinforcement Fine-Tuning of Vision-Language Planners via Video World Models
  7. 2025VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learning
  8. 2024Vision-Language Models Provide Promptable Representations for Reinforcement Learning