# Foundations for Transfer in Reinforcement Learning: A Taxonomy of Knowledge Modalities

arXiv: https://arxiv.org/abs/2312.01939 · HTML: https://mlsanity.com/paper/2312.01939/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Dec 2023.

## Built on by (most cited first, 6 of 6)

- [Towards Generalist Robot Learning from Internet Video: A Survey](https://mlsanity.com/paper/2404.19664/) (2024)
- [Imitating Language via Scalable Inverse Reinforcement Learning](https://mlsanity.com/paper/2409.01369/) (2024)
- [Learning Robot Soccer from Egocentric Vision with Deep Reinforcement Learning](https://mlsanity.com/paper/2405.02425/) (2024)
- [Provably Efficient Reward Transfer in Reinforcement Learning with Discrete Markov Decision Processes](https://mlsanity.com/paper/2503.13414/) (2025)
- [Value from Observations: Towards Large-Scale Imitation Learning via Self-Improvement](https://mlsanity.com/paper/2507.06701/) (2025)
- [An Embodied Companion for Visual Storytelling](https://mlsanity.com/paper/2603.05511/) (2026)

## Abstract (arXiv)

Contemporary artificial intelligence systems exhibit rapidly growing abilities accompanied by the growth of required resources, expansive datasets and corresponding investments into computing infrastructure. Although earlier successes predominantly focus on constrained settings, recent strides in fundamental research and applications aspire to create increasingly general systems. This evolving landscape presents a dual panorama of opportunities and challenges in refining the generalisation and transfer of knowledge - the extraction from existing sources and adaptation as a comprehensive foundation for tackling new problems. Within the domain of reinforcement learning (RL), the representation of knowledge manifests through various modalities, including dynamics and reward models, value functions, policies, and the original data. This taxonomy systematically targets these modalities and frames its discussion based on their inherent properties and alignment with different objectives and mechanisms for transfer. Where possible, we aim to provide coarse guidance delineating approaches which address requirements such as limiting environment interactions, maximising computational efficiency, and enhancing generalisation across varying axes of change. Finally, we analyse reasons contributing to the prevalence or scarcity of specific forms of transfer, the inherent potential behind pushing these frontiers, and underscore the significance of transitioning from designed to learned transfer.