mlsanityData to 14 Oct 2026

Text-to-Decision Agent: Offline Meta-Reinforcement Learning from Natural Language Supervision

arXiv 2504.15046 ↗First posted 21 Apr 2025cs.AIcited by 5

On the map: Offline RL and imitation learning from data → Decision transformers and sequence modeling

Abstract (arXiv)

Offline meta-RL usually tackles generalization by inferring task beliefs from high-quality samples or warmup explorations. The restricted form limits their generality and usability since these supervision signals are expensive and even infeasible to acquire in advance for unseen tasks. Learning directly from the raw text about decision tasks is a promising alternative to leverage a much broader source of supervision. In the paper, we propose Text-to-Decision Agent (T2DA), a simple and scalable framework that supervises offline meta-RL with natural language. We first introduce a generalized world model to encode multi-task decision data into a dynamics-aware embedding space. Then, inspired by CLIP, we predict which textual description goes with which decision embedding, effectively bridging their semantic gap via contrastive language-decision pre-training and aligning the text embeddings to comprehend the environment dynamics. After training the text-conditioned generalist policy, the agent can directly realize zero-shot text-to-decision generation in response to language instructions. Comprehensive experiments on MuJoCo and Meta-World benchmarks show that T2DA facilitates high-capacity zero-shot generalization and outperforms various types of baselines. Our code is available at textcolormagentahrefhttps://github.com/NJU-RL/T2DAhttps://github.com/NJU-RL/T2DA.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Decision transformers and sequence modeling, one of 13 directions in Offline RL and imitation learning from data. Builds on 54 of its 71 references that are in the map. First posted on arXiv 21 Apr 2025.

Builds on

54 of 71 references.

Closest ancestors · 9

Same direction, newest first

  1. 2024In-Context Decision Transformer: Reinforcement Learning via Hierarchical Chain-of-Thought31 May
  2. 2024Knowledgeable Agents by Offline Reinforcement Learning from Large Language Model Rollouts14 Apr
  3. 2024Towards an Information Theoretic Framework of Context-Based Offline Meta-Reinforcement Learning4 Feb
  4. 2023Context Shift Reduction for Offline Meta-Reinforcement Learning7 Nov
  5. 2023Unleashing the Power of Pre-trained Language Models for Offline Reinforcement Learning31 Oct
  6. 2022Prompting Decision Transformer for Few-Shot Policy Generalization27 Jun
  7. 2022Robust Task Representations for Offline Meta-Reinforcement Learning via Contrastive Learning21 Jun
  8. 2021Decision Transformer: Reinforcement Learning via Sequence Modeling2 Jun
  9. 2020Offline Meta-Reinforcement Learning with Advantage Weighting13 Aug

Other references · 45

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  8. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  9. 2022Scalable Diffusion Models with Transformerscited by 6,054
  10. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  11. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  12. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
33 more
  1. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  2. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  3. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  4. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  5. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  6. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  7. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  8. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,427
  9. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  10. 2023Training Diffusion Models with Reinforcement Learningcited by 982
  11. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 523
  12. 2022Is Conditional Generative Modeling all you need for Decision-Making?cited by 516
  13. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 503
  14. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 389
  15. 2023Reward Design with Language Modelscited by 261
  16. 2022In-context Reinforcement Learning with Algorithm Distillationcited by 190
  17. 2023Vision-Language Models are Zero-Shot Reward Models for Reinforcement Learningcited by 165
  18. 2023Text2Reward: Reward Shaping with Language Models for Reinforcement Learningcited by 160
  19. 2024Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learningcited by 158
  20. 2023Supervised Pretraining Can Learn In-Context Reinforcement Learningcited by 139
  21. 2024Robust CLIP: Unsupervised Adversarial Fine-Tuning of Vision Embeddings for Robust Large Vision-Language Modelscited by 120
  22. 2024HumanoidBench: Simulated Humanoid Benchmark for Whole-Body Locomotion and Manipulationcited by 112
  23. 2023Large Language Models as Generalizable Policies for Embodied Taskscited by 86
  24. 2024BAKU: An Efficient Transformer for Multi-Task Policy Learningcited by 80
  25. 2023MetaDiffuser: Diffusion Model as Conditional Planner for Offline Meta-RLcited by 68
  26. 2020Human Instruction-Following with Deep Reinforcement Learning via Transfer-Learning from Textcited by 59
  27. 2023AMAGO: Scalable In-Context Reinforcement Learning for Adaptive Agentscited by 41
  28. 2023Learning to Modulate pre-trained Models in RLcited by 28
  29. 2024LLM-Empowered State Representation for Reinforcement Learningcited by 28
  30. 2024FuRL: Visual-Language Models as Fuzzy Rewards for Reinforcement Learningcited by 26
  31. 2020Lifelong Incremental Reinforcement Learning with Online Bayesian Inferencecited by 23
  32. 2024Meta-DT: Offline Meta-RL as Conditional Sequence Modeling with World Model Disentanglementcited by 17
  33. 2022A Dirichlet Process Mixture of Robust Task Models for Scalable Lifelong Reinforcement Learningcited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 1.

5 papers.

  1. 2025Diversity-Incentivized Exploration for Versatile Reasoningcited by 28
  2. 2025Mixture-of-Experts Meets In-Context Reinforcement Learningcited by 8
  3. 2025Scalable In-Context Q-Learningcited by 5
  4. 2025Aligning Large Vision-Language Models by Deep Reinforcement Learning and Direct Preference Optimizationcited by 1
  5. 2026Text-to-3D Policy: Fine-Grained Language-Behavior Alignment for Unseen Specification Generalizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Reinforcement Learning from Meta-Evaluation: Aligning Language Models Without Ground-Truth Labels
  2. 2024MORE-3S:Multimodal-based Offline Reinforcement Learning with Shared Semantic Spaces
  3. 2024MetaReflection: Learning Instructions for Language Agents using Past Reflections
  4. 2022Contextual Transformer for Offline Meta Reinforcement Learning
  5. 2026ASPECT:Analogical Semantic Policy Execution via Language Conditioned Transfer
  6. 2025LLM-Driven Policy Diffusion: Enhancing Generalization in Offline Reinforcement Learning
  7. 2024RLZero: Direct Policy Inference from Language Without In-Domain Supervision
  8. 2024AMAGO-2: Breaking the Multi-Task Barrier in Meta-Reinforcement Learning with Transformers