# Similarity-based Knowledge Transfer for Cross-Domain Reinforcement Learning

arXiv: https://arxiv.org/abs/2312.03764 · HTML: https://mlsanity.com/paper/2312.03764/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 57 references that are in the map.
- First posted on arXiv 5 Dec 2023.

## Builds on (most cited first, 19 of 19)

- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [Policy Distillation](https://mlsanity.com/paper/1511.06295/) (2015)
- [Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learning](https://mlsanity.com/paper/2101.05265/) (2021)
- [Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robots](https://mlsanity.com/paper/2103.14295/) (2021)
- [Learning Invariant Feature Spaces to Transfer Skills with Reinforcement Learning](https://mlsanity.com/paper/1703.02949/) (2017)
- [Hidden Parameter Markov Decision Processes: A Semiparametric Regression Approach for Discovering Latent Task Parametrizations](https://mlsanity.com/paper/1308.3513/) (2013)
- [MushroomRL: Simplifying Reinforcement Learning Research](https://mlsanity.com/paper/2001.01102/) (2020)
- [Learning Cross-Domain Correspondence for Control with Dynamics Cycle-Consistency](https://mlsanity.com/paper/2012.09811/) (2020)
- [Cross-domain Imitation from Observations](https://mlsanity.com/paper/2105.10037/) (2021)
- [RLDS: an Ecosystem to Generate, Share and Use Datasets in Reinforcement Learning](https://mlsanity.com/paper/2111.02767/) (2021)
- [StarCraft Micromanagement with Reinforcement Learning and Curriculum Transfer Learning](https://mlsanity.com/paper/1804.00810/) (2018)
- [Lipschitz Lifelong Reinforcement Learning](https://mlsanity.com/paper/2001.05411/) (2020)
- [Learning Task-relevant Representations for Generalization via Characteristic Functions of Reward Sequence Distributions](https://mlsanity.com/paper/2205.10218/) (2022)
- [Multi-View Reinforcement Learning](https://mlsanity.com/paper/1910.08285/) (2019)
- [Mutual Information Based Knowledge Transfer Under State-Action Dimension Mismatch](https://mlsanity.com/paper/2006.07041/) (2020)
- [Knowledge-Based Hierarchical POMDPs for Task Planning](https://mlsanity.com/paper/2103.10642/) (2021)
- [Efficient Embedding of Semantic Similarity in Control Policies via Entangled Bisimulation](https://mlsanity.com/paper/2201.12300/) (2022)

## Built on by (most cited first, 4 of 4)

- [Hybrid Transfer Reinforcement Learning: Provable Sample Efficiency from Shifted-Dynamics Data](https://mlsanity.com/paper/2411.03810/) (2024)
- [Knowledge Transfer for Cross-Domain Reinforcement Learning: A Systematic Review](https://mlsanity.com/paper/2404.17687/) (2024)
- [A Unified Algorithmic Framework for Hybrid Reinforcement Learning in Tabular MDPs with Shifted Transition Dynamics](https://mlsanity.com/paper/2607.25207/) (2026)
- [Crossing the Cyber Divide: Sim-to-Sim and Sim-to-Real Transfer for RL Agents](https://mlsanity.com/paper/2610.00759/) (2026)

## Abstract (arXiv)

Transferring knowledge in cross-domain reinforcement learning is a challenging setting in which learning is accelerated by reusing knowledge from a task with different observation and/or action space. However, it is often necessary to carefully select the source of knowledge for the receiving end to benefit from the transfer process. In this article, we study how to measure the similarity between cross-domain reinforcement learning tasks to select a source of knowledge that will improve the performance of the learning agent. We developed a semi-supervised alignment loss to match different spaces with a set of encoder-decoders, and use them to measure similarity and transfer policies across tasks. In comparison to prior works, our method does not require data to be aligned, paired or collected by expert policies. Experimental results, on a set of varied Mujoco control tasks, show the robustness of our method in effectively selecting and transferring knowledge, without the supervision of a tailored set of source tasks.