mlsanityData to 7 Oct 2026

GP-Tree: A Gaussian Process Classifier for Few-Shot Incremental Learning

arXiv 2102.07868 ↗First posted 15 Feb 2021cs.LGcited by 26

Not on the map. Why?

Abstract (arXiv)

Gaussian processes (GPs) are non-parametric, flexible, models that work well in many tasks. Combining GPs with deep learning methods via deep kernel learning (DKL) is especially compelling due to the strong representational power induced by the network. However, inference in GPs, whether with or without DKL, can be computationally challenging on large datasets. Here, we propose GP-Tree, a novel method for multi-class classification with Gaussian processes and DKL. We develop a tree-based hierarchical model in which each internal node of the tree fits a GP to the data using the P'olya Gamma augmentation scheme. As a result, our method scales well with both the number of classes and data size. We demonstrate the effectiveness of our method against other Gaussian process training baselines, and we show how our general GP approach achieves improved accuracy on standard incremental few-shot learning benchmarks.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Builds on 39 of its 68 references that are in the map. First posted on arXiv 15 Feb 2021.

Builds on

39 of 68 references.

References · 39

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  4. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  5. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  6. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  7. 2016Matching Networks for One Shot Learningcited by 2,246
  8. 2016Progressive Neural Networkscited by 1,332
  9. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 882
  10. 2018Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigencecited by 734
  11. 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 711
  12. 2017Lifelong Learning with Dynamically Expandable Networkscited by 689
27 more
  1. 2018Overcoming catastrophic forgetting with hard attention to the taskcited by 672
  2. 2018End-to-End Incremental Learningcited by 596
  3. 2018Progress & Compress: A scalable framework for continual learningcited by 516
  4. 2015Deep Kernel Learningcited by 440
  5. 2018Dynamic Few-Shot Visual Learning without Forgettingcited by 385
  6. 2017Feature Generating Networks for Zero-Shot Learningcited by 257
  7. 2020Few-Shot Class-Incremental Learningcited by 249
  8. 2017Overcoming Catastrophic Forgetting by Incremental Moment Matchingcited by 243
  9. 2020Semantic Drift Compensation for Class-Incremental Learningcited by 241
  10. 2014Scalable Variational Gaussian Process Classificationcited by 226
  11. 2020Mnemonics Training: Multi-Class Incremental Learning without Forgettingcited by 220
  12. 2018When Gaussian Process Meets Big Data: A Review of Scalable GPscited by 205
  13. 2019Functional Regularisation for Continual Learning with Gaussian Processescited by 121
  14. 2016Stochastic Variational Deep Kernel Learningcited by 92
  15. 2014Manifold Gaussian Processes for Regressioncited by 79
  16. 2019Lifelong GAN: Continual Learning for Conditional Image Generationcited by 78
  17. 2018Memory Replay GANs: learning to generate images from new categories without forgettingcited by 76
  18. 2018Incremental Few-Shot Learning with Attention Attractor Networkscited by 65
  19. 2019Stochastic gradient Markov chain Monte Carlocited by 56
  20. 2017Adversarial Examples, Uncertainty, and Transfer Testing Robustness in Gaussian Process Hybrid Deep Networkscited by 48
  21. 2020Bayesian Few-Shot Classification with One-vs-Each P'olya-Gamma Augmented Gaussian Processescited by 32
  22. 2017Scalable Gaussian Processes with Billions of Inducing Inputs via Tensor Train Decompositioncited by 28
  23. 2018Tree-Structured Recurrent Switching Linear Dynamical Systems for Multi-Scale Modelingcited by 27
  24. 2015Dependent Multinomial Models Made Easy: Stick Breaking with the P'olya-Gamma Augmentationcited by 25
  25. 2018Efficient Gaussian Process Classification Using Polya-Gamma Data Augmentationcited by 17
  26. 2019Multi-Class Gaussian Process Classification Made Conjugate: Efficient Inference via Data Augmentationcited by 16
  27. 2019Efficient posterior sampling for high-dimensional imbalanced logistic regressioncited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 3; 2023: 9; 2024: 4; 2025: 5; 2026: 3.

26 papers.

Sort:

By area:

  1. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  2. 2022Forward Compatible Few-Shot Class-Incremental Learningcited by 125
  3. 2021Personalized Federated Learning with Gaussian Processescited by 76
  4. 2023Learning with Fantasy: Semantic-Aware Virtual Contrastive Constraint for Few-Shot Class-Incremental Learningcited by 55
  5. 2023A Survey on Few-Shot Class-Incremental Learningcited by 48
  6. 2024Bayesian Uncertainty for Gradient Aggregation in Multi-Task Learningcited by 11
  7. 2024CLOSER: Towards Better Representation Learning for Few-Shot Class-Incremental Learningcited by 11
  8. 2023FedHB: Hierarchical Bayesian Federated Learningcited by 8
  9. 2024Function-space Parameterization of Neural Networks for Sequential Learningcited by 7
  10. 2023Guided Deep Kernel Learningcited by 6
  11. 2025StruSR: Structure-Aware Symbolic Regression with Physics-Informed Taylor Guidancecited by 5
  12. 2021Incremental Meta-Learning via Episodic Replay Distillation for Few-Shot Image Recognitioncited by 3
14 more
  1. 2022Functional Ensemble Distillationcited by 3
  2. 2025Latest Advancements Towards Catastrophic Forgetting under Data Scarcity: A Comprehensive Survey on Few-Shot Class Incremental Learningcited by 3
  3. 2023Gaussian Process-Gated Hierarchical Mixtures of Expertscited by 2
  4. 2023Privacy Preserving Bayesian Federated Learning in Heterogeneous Settingscited by 2
  5. 2025Rethinking Approximate Gaussian Inference in Classificationcited by 2
  6. 2022Communication Efficient Distributed Learning over Wireless Channelscited by 1
  7. 2024PL-FSCIL: Harnessing the Power of Prompts for Few-Shot Class-Incremental Learningcited by 1
  8. 2025Few-Shot Speech Deepfake Detection Adaptation with Gaussian Processescited by 1
  9. 2025The Bayesian Approach to Continual Learning: An Overviewcited by 1
  10. 2023On the Calibration and Uncertainty with P'olya-Gamma Augmentation for Dialog Retrieval Modelscited by 0
  11. 2023Video-based Surgical Skill Assessment using Tree-based Gaussian Process Classifiercited by 0
  12. 2026PKI: Prior Knowledge-Infused Neural Network for Few-Shot Class-Incremental Learningcited by 0
  13. 2026Simplex-to-Euclidean Bijection for Conjugate and Calibrated Multiclass Gaussian Process Classificationcited by 0
  14. 2026Video-based Surgical Skill Assessment Using Dynamics-and-Uncertainty-Aware Tree-based Gaussian Process Classifiercited by 0