mlsanityData to 7 Oct 2026

Unsupervised Embedding Adaptation via Early-Stage Feature Reconstruction for Few-Shot Classification

arXiv 2106.11486 ↗First posted 22 Jun 2021cs.CVcited by 7

Not on the map. Why?

Abstract (arXiv)

We propose unsupervised embedding adaptation for the downstream few-shot classification task. Based on findings that deep neural networks learn to generalize before memorizing, we develop Early-Stage Feature Reconstruction (ESFR) -- a novel adaptation scheme with feature reconstruction and dimensionality-driven early stopping that finds generalizable features. Incorporating ESFR consistently improves the performance of baseline methods on all standard settings, including the recently proposed transductive method. ESFR used in conjunction with the transductive method further achieves state-of-the-art performance on mini-ImageNet, tiered-ImageNet, and CUB; especially with 1.2%~2.0% improvements in accuracy over the previous best performing method on 1-shot setting.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 45 of its 56 references that are in the map. First posted on arXiv 22 Jun 2021.

Builds on

45 of 56 references.

References · 45

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  3. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  4. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  5. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  6. 2016Matching Networks for One Shot Learningcited by 2,246
  7. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
  8. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  9. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,375
  10. 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,053
  11. 2017A Closer Look at Memorization in Deep Networkscited by 936
  12. 2019A Closer Look at Few-shot Classificationcited by 679
33 more
  1. 2017MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labelscited by 652
  2. 2018Meta-Learning with Latent Embedding Optimizationcited by 476
  3. 2018Meta-Learning for Semi-Supervised Few-Shot Classificationcited by 456
  4. 2020Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?cited by 435
  5. 2018TADAM: Task dependent adaptive metric for improved few-shot learningcited by 408
  6. 2019How does Disagreement Help Generalization against Label Corruption?cited by 390
  7. 2019Using Pre-Training Can Improve Model Robustness and Uncertaintycited by 373
  8. 2019S4L: Self-Supervised Semi-Supervised Learningcited by 335
  9. 2019A Baseline for Few-Shot Image Classificationcited by 298
  10. 2018Few-Shot Learning via Embedding Adaptation with Set-to-Set Functionscited by 293
  11. 2018Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionalitycited by 280
  12. 2019Intrinsic dimension of data representations in deep neural networkscited by 276
  13. 2019Cross Attention Network for Few-shot Classificationcited by 218
  14. 2018Learning to Propagate Labels: Transductive Propagation Network for Few-shot Learningcited by 208
  15. 2019SimpleShot: Revisiting Nearest-Neighbor Classification for Few-Shot Learningcited by 194
  16. 2018Dimensionality-Driven Learning with Noisy Labelscited by 173
  17. 2019Boosting Few-Shot Visual Learning with Self-Supervisioncited by 140
  18. 2019Edge-labeling Graph Neural Network for Few-shot Learningcited by 114
  19. 2019Finding Task-Relevant Features for Few-Shot Learning by Category Traversalcited by 101
  20. 2019Prototype Rectification for Few-Shot Learningcited by 96
  21. 2020Laplacian Regularized Few-Shot Learningcited by 90
  22. 2018An analytic theory of generalization dynamics and transfer learning in deep linear networkscited by 84
  23. 2020Embedding Propagation: Smoother Manifold for Few-Shot Classificationcited by 79
  24. 2019When Does Self-supervision Improve Few-shot Learning?cited by 78
  25. 2021On the geometry of generalization and memorization in deep neural networkscited by 71
  26. 2019Generating Classification Weights with GNN Denoising Autoencoders for Few-Shot Learningcited by 69
  27. 2019Generalization Guarantees for Neural Networks via Harnessing the Low-rank Structure of the Jacobiancited by 67
  28. 2019Transductive Episodic-Wise Adaptive Metric for Few-Shot Learningcited by 67
  29. 2018On the Intrinsic Dimensionality of Image Representationscited by 57
  30. 2020Empirical Bayes Transductive Meta-Learning with Synthetic Gradientscited by 48
  31. 2019How does Early Stopping Help Generalization against Label Noise?cited by 41
  32. 2021SelfMatch: Combining Contrastive Self-Supervision and Consistency for Semi-Supervised Learningcited by 35
  33. 2020TAFSSL: Task-Adaptive Feature Sub-Space Learning for few-shot classificationcited by 30

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 5.

7 papers.

  1. 2022A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunitiescited by 162
  2. 2022Test-Time Adaptation via Self-Training with Nearest Neighbor Informationcited by 64
  3. 2022Relating Regularization and Generalization through the Intrinsic Dimension of Activationscited by 9
  4. 2022Improving Few-Shot Image Classification Using Machine- and User-Generated Natural Language Descriptionscited by 6
  5. 2022Interval Bound Interpolation for Few-shot Learning with Few Taskscited by 1
  6. 2021Exploiting All Samples in Low-Resource Sentence Classification: Early Stopping and Initialization Parameterscited by 0
  7. 2021Dual Path Structural Contrastive Embeddings for Learning Novel Objectscited by 0