# CFTS-GAN: Continual Few-Shot Teacher Student for Generative Adversarial Networks

arXiv: https://arxiv.org/abs/2410.14749 · HTML: https://mlsanity.com/paper/2410.14749/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in General knowledge distillation methods, one of 12 directions in Knowledge distillation.
- Builds on 35 of its 44 references that are in the map.
- First posted on arXiv 17 Oct 2024.

## Placed in

- Knowledge distillation / [General knowledge distillation methods](https://mlsanity.com/distill/general-knowledge-distillation-methods/)

## Builds on (most cited first, 30 of 35)

- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [A Comprehensive Survey of Continual Learning: Theory, Method and Application](https://mlsanity.com/paper/2302.00487/) (2023)
- [Lifelong Learning with Dynamically Expandable Networks](https://mlsanity.com/paper/1708.01547/) (2017)
- [Which Training Methods for GANs do actually Converge?](https://mlsanity.com/paper/1801.04406/) (2018)
- [DER: Dynamically Expandable Representation for Class Incremental Learning](https://mlsanity.com/paper/2103.16788/) (2021)
- [Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2004.10956/) (2020)
- [Few-shot Image Generation via Cross-domain Correspondence](https://mlsanity.com/paper/2104.06820/) (2021)
- [Efficient parametrization of multi-domain deep neural networks](https://mlsanity.com/paper/1803.10082/) (2018)
- [Transferring GANs: generating images from limited data](https://mlsanity.com/paper/1805.01677/) (2018)
- [Patch Diffusion: Faster and More Data-Efficient Training of Diffusion Models](https://mlsanity.com/paper/2304.12526/) (2023)
- [Freeze the Discriminator: a Simple Baseline for Fine-Tuning GANs](https://mlsanity.com/paper/2002.10964/) (2020)
- [Forward Compatible Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2203.06953/) (2022)
- [Image Generation From Small Datasets via Batch Statistics Adaptation](https://mlsanity.com/paper/1904.01774/) (2019)
- [Lifelong GAN: Continual Learning for Conditional Image Generation](https://mlsanity.com/paper/1907.10107/) (2019)
- [Memory Replay GANs: learning to generate images from new categories without forgetting](https://mlsanity.com/paper/1809.02058/) (2018)
- [Generative Models from the perspective of Continual Learning](https://mlsanity.com/paper/1812.09111/) (2018)
- [Few Shot Generative Model Adaption via Relaxed Spatial Structural Alignment](https://mlsanity.com/paper/2203.04121/) (2022)
- [Continual Learning in Generative Adversarial Nets](https://mlsanity.com/paper/1705.08395/) (2017)
- [A Closer Look at Few-shot Image Generation](https://mlsanity.com/paper/2205.03805/) (2022)
- [A Survey on Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2304.08130/) (2023)
- [Adaptive Consistency Regularization for Semi-Supervised Transfer Learning](https://mlsanity.com/paper/2103.02193/) (2021)
- [Pre-trained Vision and Language Transformers Are Few-Shot Incremental Learners](https://mlsanity.com/paper/2404.02117/) (2024)
- [Compressing GANs using Knowledge Distillation](https://mlsanity.com/paper/1902.00159/) (2019)
- [Bayesian Structure Adaptation for Continual Learning](https://mlsanity.com/paper/1912.03624/) (2019)
- [CAM-GAN: Continual Adaptation Modules for Generative Adversarial Networks](https://mlsanity.com/paper/2103.04032/) (2021)

## Built on by (most cited first, 4 of 4)

- [CoLoR-GAN: Continual Few-Shot Learning with Low-Rank Adaptation in Generative Adversarial Networks](https://mlsanity.com/paper/2510.13869/) (2025)
- [Context Aware Grounded Teacher for Source Free Object Detection](https://mlsanity.com/paper/2504.15404/) (2025)
- [CCD: Continual Consistency Diffusion for Lifelong Generative Modeling](https://mlsanity.com/paper/2505.11936/) (2025)
- [Continual Learning for non-stationary regression via Memory-Efficient Replay](https://mlsanity.com/paper/2602.09720/) (2026)

## Abstract (arXiv)

Few-shot and continual learning face two well-known challenges in GANs: overfitting and catastrophic forgetting. Learning new tasks results in catastrophic forgetting in deep learning models. In the case of a few-shot setting, the model learns from a very limited number of samples (e.g. 10 samples), which can lead to overfitting and mode collapse. So, this paper proposes a Continual Few-shot Teacher-Student technique for the generative adversarial network (CFTS-GAN) that considers both challenges together. Our CFTS-GAN uses an adapter module as a student to learn a new task without affecting the previous knowledge. To make the student model efficient in learning new tasks, the knowledge from a teacher model is distilled to the student. In addition, the Cross-Domain Correspondence (CDC) loss is used by both teacher and student to promote diversity and to avoid mode collapse. Moreover, an effective strategy of freezing the discriminator is also utilized for enhancing performance. Qualitative and quantitative results demonstrate more diverse image synthesis and produce qualitative samples comparatively good to very stronger state-of-the-art models.