# CLIP-RL: Aligning Language and Policy Representations for Task Transfer in Reinforcement Learning

arXiv: https://arxiv.org/abs/2512.01616 · HTML: https://mlsanity.com/paper/2512.01616/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 8 references that are in the map.
- First posted on arXiv 1 Dec 2025.

## Builds on (most cited first, 6 of 6)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge](https://mlsanity.com/paper/2206.08853/) (2022)
- [Using Natural Language for Reward Shaping in Reinforcement Learning](https://mlsanity.com/paper/1903.02020/) (2019)
- [Learning to Understand Goal Specifications by Modelling Reward](https://mlsanity.com/paper/1806.01946/) (2018)
- [Accurately and Efficiently Interpreting Human-Robot Instructions of Varying Granularities](https://mlsanity.com/paper/1704.06616/) (2017)
- [Fast Task-Adaptation for Tasks Labeled Using Natural Language in Reinforcement Learning](https://mlsanity.com/paper/1910.04040/) (2019)

## Abstract (arXiv)

Recently, there has been an increasing need to develop agents capable of solving multiple tasks within the same environment, especially when these tasks are naturally associated with language. In this work, we propose a novel approach that leverages combinations of pre-trained (language, policy) pairs to establish an efficient transfer pipeline. Our algorithm is inspired by the principles of Contrastive Language-Image Pretraining (CLIP) in Computer Vision, which aligns representations across different modalities under the philosophy that ''two modalities representing the same concept should have similar representations.'' The central idea here is that the instruction and corresponding policy of a task represent the same concept, the task itself, in two different modalities. Therefore, by extending the idea of CLIP to RL, our method creates a unified representation space for natural language and policy embeddings. Experimental results demonstrate the utility of our algorithm in achieving faster transfer across tasks.