# Learning Adaptive Embedding Considering Incremental Class

arXiv: https://arxiv.org/abs/2008.13351 · HTML: https://mlsanity.com/paper/2008.13351/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Aug 2020.

## Built on by (most cited first, 21 of 21)

- [Exploiting Inter-sample and Inter-feature Relations in Dataset Distillation](https://mlsanity.com/paper/2404.00563/) (2024)
- [Robust Semi-supervised Learning by Wisely Leveraging Open-set Data](https://mlsanity.com/paper/2405.06979/) (2024)
- [Learning to Rebalance Multi-Modal Optimization by Adaptively Masking Subnetworks](https://mlsanity.com/paper/2404.08347/) (2024)
- [Robust Semi-Supervised Learning for Self-learning Open-World Classes](https://mlsanity.com/paper/2401.07551/) (2024)
- [Arithmetic Feature Interaction Is Necessary for Deep Tabular Learning](https://mlsanity.com/paper/2402.02334/) (2024)
- [MAP: Model Aggregation and Personalization in Federated Learning with Incomplete Classes](https://mlsanity.com/paper/2404.09232/) (2024)
- [MrTF: Model Refinery for Transductive Federated Learning](https://mlsanity.com/paper/2305.04201/) (2023)
- [Hessian Aware Low-Rank Perturbation for Order-Robust Continual Learning](https://mlsanity.com/paper/2311.15161/) (2023)
- [DP-CRE: Continual Relation Extraction via Decoupled Contrastive Learning and Memory Structure Preservation](https://mlsanity.com/paper/2403.02718/) (2024)
- [Cross-Modal Attention Alignment Network with Auxiliary Text Description for zero-shot sketch-based image retrieval](https://mlsanity.com/paper/2407.00979/) (2024)
- [Proposal Report for the 2nd SciCAP Competition 2024](https://mlsanity.com/paper/2407.01897/) (2024)
- [Domain Generalizable Knowledge Tracing via Concept Aggregation and Relation-Based Attention](https://mlsanity.com/paper/2407.02547/) (2024)
- [Self Adaptive Threshold Pseudo-labeling and Unreliable Sample Contrastive Loss for Semi-supervised Image Classification](https://mlsanity.com/paper/2407.03596/) (2024)
- [A Multihead Continual Learning Framework for Fine-Grained Fashion Image Retrieval with Contrastive Learning and Exponential Moving Average Distillation](https://mlsanity.com/paper/2603.20648/) (2026)
- [Information Fusion for Assistance Systems in Production Assessment](https://mlsanity.com/paper/2309.00157/) (2023)
- [The Solution for the CVPR 2023 1st foundation model challenge-Track2](https://mlsanity.com/paper/2403.17702/) (2024)
- [The Championship-Winning Solution for the 5th CLVISION Challenge 2024](https://mlsanity.com/paper/2406.16615/) (2024)
- [The Solution for the AIGC Inference Performance Optimization Competition](https://mlsanity.com/paper/2407.04991/) (2024)
- [The Solution for the sequential task continual learning track of the 2nd Greater Bay Area International Algorithm Competition](https://mlsanity.com/paper/2407.04996/) (2024)
- [The Solution for The PST-KDD-2024 OAG-Challenge](https://mlsanity.com/paper/2407.12827/) (2024)
- [Few-shot Class-incremental Fault Diagnosis by Preserving Class-Agnostic Knowledge with Dual-Granularity Representations](https://mlsanity.com/paper/2508.16634/) (2025)

## Abstract (arXiv)

Class-Incremental Learning (CIL) aims to train a reliable model with the streaming data, which emerges unknown classes sequentially. Different from traditional closed set learning, CIL has two main challenges: 1) Novel class detection. The initial training data only contains incomplete classes, and streaming test data will accept unknown classes. Therefore, the model needs to not only accurately classify known classes, but also effectively detect unknown classes; 2) Model expansion. After the novel classes are detected, the model needs to be updated without re-training using entire previous data. However, traditional CIL methods have not fully considered these two challenges, first, they are always restricted to single novel class detection each phase and embedding confusion caused by unknown classes. Besides, they also ignore the catastrophic forgetting of known categories in model update. To this end, we propose a Class-Incremental Learning without Forgetting (CILF) framework, which aims to learn adaptive embedding for processing novel class detection and model update in a unified framework. In detail, CILF designs to regularize classification with decoupled prototype based loss, which can improve the intra-class and inter-class structure significantly, and acquire a compact embedding representation for novel class detection in result. Then, CILF employs a learnable curriculum clustering operator to estimate the number of semantic clusters via fine-tuning the learned network, in which curriculum operator can adaptively learn the embedding in self-taught form. Therefore, CILF can detect multiple novel classes and mitigate the embedding confusion problem. Last, with the labeled streaming test data, CILF can update the network with robust regularization to mitigate the catastrophic forgetting. Consequently, CILF is able to iteratively perform novel class detection and model update.