# Encoder Based Lifelong Learning

arXiv: https://arxiv.org/abs/1704.01920 · HTML: https://mlsanity.com/paper/1704.01920/

## Key facts

- Cited by 102 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Apr 2017.

## Built on by (most cited first, 30 of 102)

- [A Comprehensive Survey of Continual Learning: Theory, Method and Application](https://mlsanity.com/paper/2302.00487/) (2023)
- [Online Continual Learning in Image Classification: An Empirical Survey](https://mlsanity.com/paper/2101.10423/) (2021)
- [Class-Incremental Learning: A Survey](https://mlsanity.com/paper/2302.03648/) (2023)
- [RL's Razor: Why Online Reinforcement Learning Forgets Less](https://mlsanity.com/paper/2509.04259/) (2025)
- [Supervised Contrastive Replay: Revisiting the Nearest Class Mean Classifier in Online Class-Incremental Continual Learning](https://mlsanity.com/paper/2103.13885/) (2021)
- [Continual Learning for Text Classification with Information Disentanglement Based Regularization](https://mlsanity.com/paper/2104.05489/) (2021)
- [Continual evaluation for lifelong learning: Identifying the stability gap](https://mlsanity.com/paper/2205.13452/) (2022)
- [Recent Advances of Continual Learning in Computer Vision: An Overview](https://mlsanity.com/paper/2109.11369/) (2021)
- [An Efficient Domain-Incremental Learning Approach to Drive in All Weather Conditions](https://mlsanity.com/paper/2204.08817/) (2022)
- [Dense Network Expansion for Class Incremental Learning](https://mlsanity.com/paper/2303.12696/) (2023)
- [LM-Cocktail: Resilient Tuning of Language Models via Model Merging](https://mlsanity.com/paper/2311.13534/) (2023)
- [Traj-MAE: Masked Autoencoders for Trajectory Prediction](https://mlsanity.com/paper/2303.06697/) (2023)
- [Graph Few-shot Class-incremental Learning](https://mlsanity.com/paper/2112.12819/) (2021)
- [Continual Learning for Blind Image Quality Assessment](https://mlsanity.com/paper/2102.09717/) (2021)
- [Multi-View Correlation Distillation for Incremental Object Detection](https://mlsanity.com/paper/2107.01787/) (2021)
- [The Challenges of Continuous Self-Supervised Learning](https://mlsanity.com/paper/2203.12710/) (2022)
- [Online Hyperparameter Optimization for Class-Incremental Learning](https://mlsanity.com/paper/2301.05032/) (2023)
- [Towards Better Plasticity-Stability Trade-off in Incremental Learning: A Simple Linear Connector](https://mlsanity.com/paper/2110.07905/) (2021)
- [Continual Neural Mapping: Learning An Implicit Scene Representation from Sequential Observations](https://mlsanity.com/paper/2108.05851/) (2021)
- [Balancing Stability and Plasticity through Advanced Null Space in Continual Learning](https://mlsanity.com/paper/2207.12061/) (2022)
- [Open-world Semantic Segmentation for LIDAR Point Clouds](https://mlsanity.com/paper/2207.01452/) (2022)
- [Lossless Adaptation of Pretrained Vision Models For Robotic Manipulation](https://mlsanity.com/paper/2304.06600/) (2023)
- [Diversifying the Mixture-of-Experts Representation for Language Models with Orthogonal Optimizer](https://mlsanity.com/paper/2310.09762/) (2023)
- [Mining Unseen Classes via Regional Objectness: A Simple Baseline for Incremental Segmentation](https://mlsanity.com/paper/2211.06866/) (2022)
- [Reinforced Continual Learning for Graphs](https://mlsanity.com/paper/2209.01556/) (2022)
- [Catastrophic Forgetting in Deep Learning: A Comprehensive Taxonomy](https://mlsanity.com/paper/2312.10549/) (2023)
- [Continual Learning in Medical Image Analysis: A Comprehensive Review of Recent Advancements and Future Prospects](https://mlsanity.com/paper/2312.17004/) (2023)
- [Continual Learning Based on OOD Detection and Task Masking](https://mlsanity.com/paper/2203.09450/) (2022)
- [Online Continual Learning with Contrastive Vision Transformer](https://mlsanity.com/paper/2207.13516/) (2022)
- [On the importance of cross-task features for class-incremental learning](https://mlsanity.com/paper/2106.11930/) (2021)

## Abstract (arXiv)

This paper introduces a new lifelong learning solution where a single model is trained for a sequence of tasks. The main challenge that vision systems face in this context is catastrophic forgetting: as they tend to adapt to the most recently seen task, they lose performance on the tasks that were learned previously. Our method aims at preserving the knowledge of the previous tasks while learning a new one by using autoencoders. For each task, an under-complete autoencoder is learned, capturing the features that are crucial for its achievement. When a new task is presented to the system, we prevent the reconstructions of the features with these autoencoders from changing, which has the effect of preserving the information on which the previous tasks are mainly relying. At the same time, the features are given space to adjust to the most recent environment as only their projection into a low dimension submanifold is controlled. The proposed system is evaluated on image classification tasks and shows a reduction of forgetting over the state-of-the-art