# Exploring Incompatible Knowledge Transfer in Few-shot Image Generation

arXiv: https://arxiv.org/abs/2304.07574 · HTML: https://mlsanity.com/paper/2304.07574/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 69 of its 96 references that are in the map.
- First posted on arXiv 15 Apr 2023.

## Builds on (most cited first, 30 of 69)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014)
- [Training Generative Adversarial Networks with Limited Data](https://mlsanity.com/paper/2006.06676/) (2020)
- [Alias-Free Generative Adversarial Networks](https://mlsanity.com/paper/2106.12423/) (2021)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [StarGAN v2: Diverse Image Synthesis for Multiple Domains](https://mlsanity.com/paper/1912.01865/) (2019)
- [Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution](https://mlsanity.com/paper/2202.10054/) (2022)
- [Network Dissection: Quantifying Interpretability of Deep Visual Representations](https://mlsanity.com/paper/1704.05796/) (2017)
- [Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks](https://mlsanity.com/paper/2102.00554/) (2021)
- [Quantifying the Carbon Emissions of Machine Learning](https://mlsanity.com/paper/1910.09700/) (2019)
- [In-Domain GAN Inversion for Real Image Editing](https://mlsanity.com/paper/2004.00049/) (2020)
- [Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration](https://mlsanity.com/paper/1811.00250/) (2018)

## Built on by (most cited first, 19 of 19)

- [Label-Only Model Inversion Attacks via Knowledge Transfer](https://mlsanity.com/paper/2310.19342/) (2023)
- [A Survey on Generative Modeling with Limited Data, Few Shots, and Zero Shot](https://mlsanity.com/paper/2307.14397/) (2023)
- [Deformable One-shot Face Stylization via DINO Semantic Guidance](https://mlsanity.com/paper/2403.00459/) (2024)
- [Image Synthesis under Limited Data: A Survey and Taxonomy](https://mlsanity.com/paper/2307.16879/) (2023)
- [Generative Artificial Intelligence Meets Synthetic Aperture Radar: A Survey](https://mlsanity.com/paper/2411.05027/) (2024)
- [On Measuring Fairness in Generative Models](https://mlsanity.com/paper/2310.19297/) (2023)
- [Few-Shot Image Generation by Conditional Relaxing Diffusion Inversion](https://mlsanity.com/paper/2407.07249/) (2024)
- [Few-shot Implicit Function Generation via Equivariance](https://mlsanity.com/paper/2501.01601/) (2025)
- [Uni-DAD: Unified Distillation and Adaptation of Diffusion Models for Few-step Few-shot Image Generation](https://mlsanity.com/paper/2511.18281/) (2025)
- [Few-shot 3D Shape Generation](https://mlsanity.com/paper/2305.11664/) (2023)
- [AdAM: Few-Shot Image Generation via Adaptation-Aware Kernel Modulation](https://mlsanity.com/paper/2307.01465/) (2023)
- [A Universal Knowledge Embedded Contrastive Learning Framework for Hyperspectral Image Classification](https://mlsanity.com/paper/2404.01673/) (2024)
- [DomainGallery: Few-shot Domain-driven Image Generation by Attribute-centric Finetuning](https://mlsanity.com/paper/2411.04571/) (2024)
- [UniHDA: A Unified and Versatile Framework for Multi-Modal Hybrid Domain Adaptation](https://mlsanity.com/paper/2401.12596/) (2024)
- [On The Relationship between Visual Anomaly-free and Anomalous Representations](https://mlsanity.com/paper/2410.06576/) (2024)
- [TAGE: Trustworthy Attribute Group Editing for Stable Few-shot Image Generation](https://mlsanity.com/paper/2410.17855/) (2024)
- [AIR: Zero-shot Generative Model Adaptation with Iterative Refinement](https://mlsanity.com/paper/2506.10895/) (2025)
- [A Turn Toward Better Alignment: Few-Shot Generative Adaptation with Equivariant Feature Rotation](https://mlsanity.com/paper/2512.21174/) (2025)
- [Few-Shot Generative Model Adaption via Identity Injection and Preservation](https://mlsanity.com/paper/2603.22965/) (2026)

## Abstract (arXiv)

Few-shot image generation (FSIG) learns to generate diverse and high-fidelity images from a target domain using a few (e.g., 10) reference samples. Existing FSIG methods select, preserve and transfer prior knowledge from a source generator (pretrained on a related domain) to learn the target generator. In this work, we investigate an underexplored issue in FSIG, dubbed as incompatible knowledge transfer, which would significantly degrade the realisticness of synthetic samples. Empirical observations show that the issue stems from the least significant filters from the source generator. To this end, we propose knowledge truncation to mitigate this issue in FSIG, which is a complementary operation to knowledge preservation and is implemented by a lightweight pruning-based method. Extensive experiments show that knowledge truncation is simple and effective, consistently achieving state-of-the-art performance, including challenging setups where the source and target domains are more distant. Project Page: yunqing-me.github.io/RICK.