# Modularization of End-to-End Learning: Case Study in Arcade Games

arXiv: https://arxiv.org/abs/1901.09895 · HTML: https://mlsanity.com/paper/1901.09895/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Jan 2019.

## Built on by (most cited first, 6 of 6)

- [UniTeam: Open Vocabulary Mobile Manipulation Challenge](https://mlsanity.com/paper/2312.08611/) (2023)
- [Critic Guided Segmentation of Rewarding Objects in First-Person Views](https://mlsanity.com/paper/2107.09540/) (2021)
- [Behavioral Cloning via Search in Video PreTraining Latent Space](https://mlsanity.com/paper/2212.13326/) (2022)
- [Planning with RL and episodic-memory behavioral priors](https://mlsanity.com/paper/2207.01845/) (2022)
- [Solving Learn-to-Race Autonomous Racing Challenge by Planning in Latent Space](https://mlsanity.com/paper/2207.01275/) (2022)
- [Towards robust and domain agnostic reinforcement learning competitions](https://mlsanity.com/paper/2106.03748/) (2021)

## Abstract (arXiv)

Complex environments and tasks pose a difficult problem for holistic end-to-end learning approaches. Decomposition of an environment into interacting controllable and non-controllable objects allows supervised learning for non-controllable objects and universal value function approximator learning for controllable objects. Such decomposition should lead to a shorter learning time and better generalisation capability. Here, we consider arcade-game environments as sets of interacting objects (controllable, non-controllable) and propose a set of functional modules that are specialized on mastering different types of interactions in a broad range of environments. The modules utilize regression, supervised learning, and reinforcement learning algorithms. Results of this case study in different Atari games suggest that human-level performance can be achieved by a learning agent within a human amount of game experience (10-15 minutes game time) when a proper decomposition of an environment or a task is provided. However, automatization of such decomposition remains a challenging problem. This case study shows how a model of a causal structure underlying an environment or a task can benefit learning time and generalization capability of the agent, and argues in favor of exploiting modular structure in contrast to using pure end-to-end learning approaches.