MORE-3S:Multimodal-based Offline Reinforcement Learning with Shared Semantic Spaces
On the map: Offline RL and imitation learning from data → Decision transformers and sequence modeling
Abstract (arXiv)
Drawing upon the intuition that aligning different modalities to the same semantic embedding space would allow models to understand states and actions more easily, we propose a new perspective to the offline reinforcement learning (RL) challenge. More concretely, we transform it into a supervised learning task by integrating multimodal and pre-trained language models. Our approach incorporates state information derived from images and action-related data obtained from text, thereby bolstering RL training performance and promoting long-term strategic thinking. We emphasize the contextual understanding of language and demonstrate how decision-making in RL can benefit from aligning states' and actions' representation with languages' representation. Our method significantly outperforms current baselines as evidenced by evaluations conducted on Atari and OpenAI Gym environments. This contributes to advancing offline RL performance and efficiency while providing a novel perspective on offline RL.Our code and data are available at https://github.com/Zheng0428/MORE_.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Decision transformers and sequence modeling, one of 13 directions in Offline RL and imitation learning from data. Builds on 38 of its 57 references that are in the map. First posted on arXiv 20 Feb 2024.
Builds on
38 of 57 references.
Closest ancestors · 4
Same direction, newest first
- 2024Read to Play (R2-Play): Decision Transformer with Multimodal Game Instruction6 Feb
- 2022Q-learning Decision Transformer: Leveraging Dynamic Programming for Conditional Sequence Modelling in Offline RL8 Sep
- 2021Offline Reinforcement Learning as One Big Sequence Modeling Problem3 Jun
- 2021Decision Transformer: Reinforcement Learning via Sequence Modeling2 Jun
Other references · 34
- 2017Attention Is All You Needcited by 59,932
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2014Microsoft COCO: Common Objects in Contextcited by 17,498
- 2015Fast R-CNNcited by 2,930
- 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
- 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
- 2015VQA: Visual Question Answeringcited by 2,489
- 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
- 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
- 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
- 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,548
22 more
- 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
- 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
- 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,310
- 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
- 2021A Minimalist Approach to Offline Reinforcement Learningcited by 882
- 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 750
- 2021Multimodal Few-Shot Learning with Frozen Language Modelscited by 657
- 2019Behavior Regularized Offline Reinforcement Learningcited by 654
- 2020MOPO: Model-based Offline Policy Optimizationcited by 630
- 2020MOReL : Model-Based Offline Reinforcement Learningcited by 582
- 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 469
- 2023Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agentscited by 396
- 2021Offline Reinforcement Learning with Fisher Divergence Critic Regularizationcited by 237
- 2022MeMViT: Memory-Augmented Multiscale Vision Transformer for Efficient Long-Term Video Recognitioncited by 189
- 2021Pretrained Transformers as Universal Computation Enginescited by 162
- 2021Uncertainty Weighted Actor-Critic for Offline Reinforcement Learningcited by 125
- 2019Training Agents using Upside-Down Reinforcement Learningcited by 106
- 2021Believe What You See: Implicit Constraint Approach for Offline Multi-Agent Reinforcement Learningcited by 78
- 2020Implicit Class-Conditioned Domain Alignment for Unsupervised Domain Adaptationcited by 60
- 2023Learning to Modulate pre-trained Models in RLcited by 28
- 2021StARformer: Transformer with State-Action-Reward Representations for Visual Reinforcement Learningcited by 27
- 2020AlignTTS: Efficient Feed-Forward Text-to-Speech System without Explicit Alignmentcited by 20
Built on by
Citing papers per year: 2024: 1.
1 paper.
- 2024GameGen-X: Interactive Open-world Game Video Generationcited by 124
- 2024GameGen-X: Interactive Open-world Game Video Generation1 Nov 2024
Similar, not cited
Close in wording, no citation either way
- 2025The Evolving Landscape of LLM- and VLM-Integrated Reinforcement Learning
- 2023Unleashing the Power of Pre-trained Language Models for Offline Reinforcement Learning
- 2022A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problems
- 2025Reinforced MLLM: A Survey on RL-Based Reasoning in Multimodal Large Language Models
- 2026Leveraging Offline Supervision for Efficient and Generalizable Reinforcement Learning in Large-Scale Vision-Language-Action Models
- 2026VLGOR: Visual-Language Knowledge Guided Offline Reinforcement Learning for Generalizable Agents
- 2026STO-RL: Offline RL under Sparse Rewards via LLM-Guided Subgoal Temporal Order
- 2025ImagineBench: Evaluating Reinforcement Learning with Large Language Model Rollouts