# CLIC: Curriculum Learning and Imitation for object Control in non-rewarding environments

arXiv: https://arxiv.org/abs/1901.09720 · HTML: https://mlsanity.com/paper/1901.09720/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Jan 2019.

## Built on by (most cited first, 10 of 10)

- [Curriculum Learning: A Survey](https://mlsanity.com/paper/2101.10382/) (2021)
- [Learning to Reason at the Frontier of Learnability](https://mlsanity.com/paper/2502.12272/) (2025)
- [SkiLD: Unsupervised Skill Discovery Guided by Factor Interactions](https://mlsanity.com/paper/2410.18416/) (2024)
- [DisTop: Discovering a Topological representation to learn diverse and rewarding skills](https://mlsanity.com/paper/2106.03853/) (2021)
- [Intrinsically Motivated Open-Ended Multi-Task Learning Using Transfer Learning to Discover Task Hierarchy](https://mlsanity.com/paper/2102.09854/) (2021)
- [Help Me Explore: Minimal Social Interventions for Graph-Based Autotelic Agents](https://mlsanity.com/paper/2202.05129/) (2022)
- [Towards Teachable Autotelic Agents](https://mlsanity.com/paper/2105.11977/) (2021)
- [Pragmatically Learning from Pedagogical Demonstrations in Multi-Goal Environments](https://mlsanity.com/paper/2206.04546/) (2022)
- [A Comprehensive Study of Privacy Risks in Curriculum Learning](https://mlsanity.com/paper/2310.10124/) (2023)
- [The intrinsic motivation of reinforcement and imitation learning for sequential tasks](https://mlsanity.com/paper/2412.20573/) (2024)

## Abstract (arXiv)

In this paper we study a new reinforcement learning setting where the environment is non-rewarding, contains several possibly related objects of various controllability, and where an apt agent Bob acts independently, with non-observable intentions. We argue that this setting defines a realistic scenario and we present a generic discrete-state discrete-action model of such environments. To learn in this environment, we propose an unsupervised reinforcement learning agent called CLIC for Curriculum Learning and Imitation for Control. CLIC learns to control individual objects in its environment, and imitates Bob's interactions with these objects. It selects objects to focus on when training and imitating by maximizing its learning progress. We show that CLIC is an effective baseline in our new setting. It can effectively observe Bob to gain control of objects faster, even if Bob is not explicitly teaching. It can also follow Bob when he acts as a mentor and provides ordered demonstrations. Finally, when Bob controls objects that the agent cannot, or in presence of a hierarchy between objects in the environment, we show that CLIC ignores non-reproducible and already mastered interactions with objects, resulting in a greater benefit from imitation.