# SHELS: Exclusive Feature Sets for Novelty Detection and Continual Learning Without Class Boundaries

arXiv: https://arxiv.org/abs/2206.13720 · HTML: https://mlsanity.com/paper/2206.13720/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 54 references that are in the map.
- First posted on arXiv 28 Jun 2022.

## Builds on (most cited first, 27 of 27)

- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks](https://mlsanity.com/paper/1807.03888/) (2018)
- [Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks](https://mlsanity.com/paper/1706.02690/) (2017)
- [Continual Learning with Deep Generative Replay](https://mlsanity.com/paper/1705.08690/) (2017)
- [Efficient Lifelong Learning with A-GEM](https://mlsanity.com/paper/1812.00420/) (2018)
- [Learning Structured Sparsity in Deep Neural Networks](https://mlsanity.com/paper/1608.03665/) (2016)
- [Towards Open Set Deep Networks](https://mlsanity.com/paper/1511.06233/) (2015)
- [PathNet: Evolution Channels Gradient Descent in Super Neural Networks](https://mlsanity.com/paper/1701.08734/) (2017)
- [Gradient Projection Memory for Continual Learning](https://mlsanity.com/paper/2103.09762/) (2021)
- [How to train your MAML](https://mlsanity.com/paper/1810.09502/) (2018)
- [Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgetting](https://mlsanity.com/paper/1904.00310/) (2019)
- [Towards Open World Recognition](https://mlsanity.com/paper/1412.5687/) (2014)
- [Continual Learning with Node-Importance based Adaptive Group Sparse Regularization](https://mlsanity.com/paper/2003.13726/) (2020)
- [Continual Learning in Low-rank Orthogonal Subspaces](https://mlsanity.com/paper/2010.11635/) (2020)
- [Efficient Continual Learning with Modular Networks and Task-Driven Priors](https://mlsanity.com/paper/2012.12631/) (2020)
- [Flattening Sharpness for Dynamic Gradient Projection Memory Benefits Continual Learning](https://mlsanity.com/paper/2110.04593/) (2021)
- [Deep Multi-task Representation Learning: A Tensor Factorisation Approach](https://mlsanity.com/paper/1605.06391/) (2016)
- [Continual Learning via Local Module Composition](https://mlsanity.com/paper/2111.07736/) (2021)
- [Cosine Normalization: Using Cosine Similarity Instead of Dot Product in Neural Networks](https://mlsanity.com/paper/1702.05870/) (2017)
- [HOUDINI: Lifelong Learning as Program Synthesis](https://mlsanity.com/paper/1804.00218/) (2018)
- [Lifelong Learning of Compositional Structures](https://mlsanity.com/paper/2007.07732/) (2020)

## Built on by (most cited first, 7 of 7)

- [Open-World Continual Learning: Unifying Novelty Detection and Continual Learning](https://mlsanity.com/paper/2304.10038/) (2023)
- [Advancing Prompt-Based Methods for Replay-Independent General Continual Learning](https://mlsanity.com/paper/2503.00677/) (2025)
- [A Metacognitive Approach to Out-of-Distribution Detection for Segmentation](https://mlsanity.com/paper/2311.07578/) (2023)
- [Policy Search, Retrieval, and Composition via Task Similarity in Collaborative Agentic Systems](https://mlsanity.com/paper/2506.05577/) (2025)
- [Continual Improvement of Threshold-Based Novelty Detection](https://mlsanity.com/paper/2309.02551/) (2023)
- [Learning After Model Deployment](https://mlsanity.com/paper/2510.17160/) (2025)
- [MAND: Modality-Aware Novelty Detection for Open-World Egocentric Activity Recognition](https://mlsanity.com/paper/2603.16970/) (2026)

## Abstract (arXiv)

While deep neural networks (DNNs) have achieved impressive classification performance in closed-world learning scenarios, they typically fail to generalize to unseen categories in dynamic open-world environments, in which the number of concepts is unbounded. In contrast, human and animal learners have the ability to incrementally update their knowledge by recognizing and adapting to novel observations. In particular, humans characterize concepts via exclusive (unique) sets of essential features, which are used for both recognizing known classes and identifying novelty. Inspired by natural learners, we introduce a Sparse High-level-Exclusive, Low-level-Shared feature representation (SHELS) that simultaneously encourages learning exclusive sets of high-level features and essential, shared low-level features. The exclusivity of the high-level features enables the DNN to automatically detect out-of-distribution (OOD) data, while the efficient use of capacity via sparse low-level features permits accommodating new knowledge. The resulting approach uses OOD detection to perform class-incremental continual learning without known class boundaries. We show that using SHELS for novelty detection results in statistically significant improvements over state-of-the-art OOD detection approaches over a variety of benchmark datasets. Further, we demonstrate that the SHELS model mitigates catastrophic forgetting in a class-incremental learning setting,enabling a combined novelty detection and accommodation framework that supports learning in open-world settings