mlsanityData to 14 Oct 2026

New Perspective on Progressive GANs Distillation for One-class Novelty Detection

arXiv 2109.07295 ↗First posted 15 Sep 2021cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

One-class novelty detection is conducted to identify anomalous instances, with different distributions from the expected normal instances. In this paper, the Generative Adversarial Network based on the Encoder-Decoder-Encoder scheme (EDE-GAN) achieves state-of-the-art performance. The two factors bellow serve the above purpose: 1) The EDE-GAN calculates the distance between two latent vectors as the anomaly score, which is unlike the previous methods by utilizing the reconstruction error between images. 2) The model obtains best results when the batch size is set to 1. To illustrate their superiority, we design a new GAN architecture, and compare performances according to different batch sizes. Moreover, with experimentation leads to discovery, our result implies there is also evidence of just how beneficial constraint on the latent space are when engaging in model training. In an attempt to learn compact and fast models, we present a new technology, Progressive Knowledge Distillation with GANs (P-KDGAN), which connects two standard GANs through the designed distillation loss. Two-step progressive learning continuously augments the performance of student GANs with improved results over single-step approach. Our experimental results on CIFAR-10, MNIST, and FMNIST datasets illustrate that P-KDGAN improves the performance of the student GAN by 2.44%, 1.77%, and 1.73% when compressing the computationat ratios of 24.45:1, 311.11:1, and 700:1, respectively.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 31 of its 57 references that are in the map. First posted on arXiv 15 Sep 2021.

Builds on

31 of 57 references.

References · 31

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  4. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  5. 2013Auto-Encoding Variational Bayescited by 4,053
  6. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,054
  7. 2014FitNets: Hints for Thin Deep Netscited by 1,653
  8. 2017Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discoverycited by 566
  9. 2019Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detectioncited by 564
  10. 2021CutPaste: Self-Supervised Learning for Anomaly Detection and Localizationcited by 531
  11. 2018GANomaly: Semi-Supervised Anomaly Detection via Adversarial Trainingcited by 409
  12. 2016Adversarial Feature Learningcited by 350
19 more
  1. 2017A Survey of Model Compression and Acceleration for Deep Neural Networkscited by 301
  2. 2019Deep Semi-Supervised Anomaly Detectioncited by 291
  3. 2020Multiresolution Knowledge Distillation for Anomaly Detectioncited by 285
  4. 2018Deep Anomaly Detection Using Geometric Transformationscited by 269
  5. 2019OCGAN: One-class Novelty Detection Using GANs with Constrained Latent Representationscited by 199
  6. 2019Deep Anomaly Detection with Deviation Networkscited by 153
  7. 2018Latent Space Autoregression for Novelty Detectioncited by 149
  8. 2018Efficient GAN-Based Anomaly Detectioncited by 122
  9. 2019Skip-GANomaly: Skip Connected and Adversarially Trained Encoder-Decoder Anomaly Detectioncited by 100
  10. 2020Self-trained Deep Ordinal Regression for End-to-End Video Anomaly Detectioncited by 83
  11. 2018Anomaly Detection using One-Class Neural Networkscited by 82
  12. 2018Learning Deep Features for One-Class Classificationcited by 76
  13. 2019Deep Weakly-supervised Anomaly Detectioncited by 59
  14. 2021Rethinking "Batch" in BatchNormcited by 45
  15. 2019Fence GAN: Towards Better Anomaly Detectioncited by 32
  16. 2019Anomaly Detection with Adversarial Dual Autoencoderscited by 16
  17. 2019Attention Guided Anomaly Localization in Imagescited by 9
  18. 2018KTAN: Knowledge Transfer Adversarial Networkcited by 4
  19. 2019Fixing Bias in Reconstruction-based Anomaly Detection with Lipschitz Discriminatorscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2022Fake It Till You Make It: Towards Accurate Near-Distribution Novelty Detection
  2. 2022Lightning Fast Video Anomaly Detection via Adversarial Knowledge Distillation
  3. 2023KD-DLGAN: Data Limited Image Generation via Knowledge Distillation
  4. 2025Scale-Aware Contrastive Reverse Distillation for Unsupervised Medical Anomaly Detection
  5. 2023Towards Universal Fake Image Detectors that Generalize Across Generative Models
  6. 2026Improving Diversity in Black-box Few-shot Knowledge Distillation
  7. 2025SDAKD: Student Discriminator Assisted Knowledge Distillation for Super-Resolution Generative Adversarial Networks
  8. 2025Generative Adversarial Networks Bridging Art and Machine Intelligence