# Towards Lifelong Self-Supervision For Unpaired Image-to-Image Translation

arXiv: https://arxiv.org/abs/2004.00161 · HTML: https://mlsanity.com/paper/2004.00161/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Mar 2020.

## Built on by (most cited first, 2 of 2)

- [Deep Translation Prior: Test-time Training for Photorealistic Style Transfer](https://mlsanity.com/paper/2112.06150/) (2021)
- [Reviewing continual learning from the perspective of human-level intelligence](https://mlsanity.com/paper/2111.11964/) (2021)

## Abstract (arXiv)

Unpaired Image-to-Image Translation (I2IT) tasks often suffer from lack of data, a problem which self-supervised learning (SSL) has recently been very popular and successful at tackling. Leveraging auxiliary tasks such as rotation prediction or generative colorization, SSL can produce better and more robust representations in a low data regime. Training such tasks along an I2IT task is however computationally intractable as model size and the number of task grow. On the other hand, learning sequentially could incur catastrophic forgetting of previously learned tasks. To alleviate this, we introduce Lifelong Self-Supervision (LiSS) as a way to pre-train an I2IT model (e.g., CycleGAN) on a set of self-supervised auxiliary tasks. By keeping an exponential moving average of past encoders and distilling the accumulated knowledge, we are able to maintain the network's validation performance on a number of tasks without any form of replay, parameter isolation or retraining techniques typically used in continual learning. We show that models trained with LiSS perform better on past tasks, while also being more robust than the CycleGAN baseline to color bias and entity entanglement (when two entities are very close).