mlsanityData to 7 Oct 2026

Foundation Models for Semantic Novelty in Reinforcement Learning

arXiv 2211.04878 ↗First posted 9 Nov 2022cs.LG cs.AIcited by 9

Not on the map. Why?

Abstract (arXiv)

Effectively exploring the environment is a key challenge in reinforcement learning (RL). We address this challenge by defining a novel intrinsic reward based on a foundation model, such as contrastive language image pretraining (CLIP), which can encode a wealth of domain-independent semantic visual-language knowledge about the world. Specifically, our intrinsic reward is defined based on pre-trained CLIP embeddings without any fine-tuning or learning on the target RL task. We demonstrate that CLIP-based intrinsic rewards can drive exploration towards semantically meaningful states and outperform state-of-the-art methods in challenging sparse-reward procedurally-generated environments.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 30 of its 39 references that are in the map. First posted on arXiv 9 Nov 2022.

Builds on

30 of 39 references.

References · 30

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  4. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  5. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 990
  6. 2018Exploration by Random Network Distillationcited by 966
  7. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
  8. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  9. 2021CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrievalcited by 645
  10. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 632
  11. 2021CLIPort: What and Where Pathways for Robotic Manipulationcited by 617
  12. 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 564
18 more
  1. 2016#Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learningcited by 307
  2. 2018Large-Scale Study of Curiosity-Driven Learningcited by 277
  3. 2017Noisy Networks for Explorationcited by 262
  4. 2020ObjectNav Revisited: On Evaluation of Embodied Agents Navigating to Objectscited by 240
  5. 2017Concrete Dropoutcited by 227
  6. 2021CLIP2Video: Mastering Video-Text Retrieval via Image CLIPcited by 223
  7. 2020Never Give Up: Learning Directed Exploration Strategiescited by 210
  8. 2021Simple but Effective: CLIP Embeddings for Embodied AIcited by 191
  9. 2015Incentivizing Exploration In Reinforcement Learning With Deep Predictive Modelscited by 154
  10. 2020RIDE: Rewarding Impact-Driven Exploration for Procedurally-Generated Environmentscited by 135
  11. 2014Generalization and Exploration via Randomized Value Functionscited by 112
  12. 2018Count-Based Exploration with the Successor Representationcited by 112
  13. 2017The Uncertainty Bellman Equation and Explorationcited by 81
  14. 2019A survey on intrinsic motivation in reinforcement learningcited by 79
  15. 2022Semantic Exploration from Language Abstractions and Pretrained Representationscited by 51
  16. 2022Improving Intrinsic Exploration with Language Abstractionscited by 49
  17. 2017Count-Based Exploration in Feature Space for Reinforcement Learningcited by 48
  18. 2018DORA The Explorer: Directed Outreaching Reinforcement Action-Selectioncited by 27

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 3; 2025: 1; 2026: 2.

9 papers.

  1. 2023Grounding Large Language Models in Interactive Environments with Online Reinforcement Learningcited by 193
  2. 2024The Essential Role of Causality in Foundation World Models for Embodied AIcited by 32
  3. 2023Semantic HELM: A Human-Readable Memory for Reinforcement Learningcited by 17
  4. 2024YOLO-MARL: You Only LLM Once for Multi-Agent Reinforcement Learningcited by 9
  5. 2023Semantically Aligned Task Decomposition in Multi-Agent Reinforcement Learningcited by 8
  6. 2024Fostering Intrinsic Motivation in Reinforcement Learning with Pretrained Foundation Modelscited by 0
  7. 2025Toward Explainable Offline RL: Analyzing Representations in Intrinsically Motivated Decision Transformerscited by 0
  8. 2026VLM-Guided Experience Replaycited by 0
  9. 2026Right Makes Might: Aligning Verified Hidden States Empowers RL Reasoningcited by 0