mlsanityData to 7 Oct 2026

FlyPrompt: Brain-Inspired Random-Expanded Routing with Temporal-Ensemble Experts for General Continual Learning

arXiv 2602.01976 ↗First posted 2 Feb 2026cs.LG cs.AI cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

General continual learning (GCL) challenges intelligent systems to learn from single-pass, non-stationary data streams without clear task boundaries. While recent advances in continual parameter-efficient tuning (PET) of pretrained models show promise, they typically rely on multiple training epochs and explicit task cues, limiting their effectiveness in GCL scenarios. Moreover, existing methods often lack targeted design and fail to address two fundamental challenges in continual PET: how to allocate expert parameters to evolving data distributions, and how to improve their representational capacity under limited supervision. Inspired by the fruit fly's hierarchical memory system characterized by sparse expansion and modular ensembles, we propose FlyPrompt, a brain-inspired framework that decomposes GCL into two subproblems: expert routing and expert competence improvement. FlyPrompt introduces a randomly expanded analytic router for instance-level expert activation and a temporal ensemble of output heads to dynamically adapt decision boundaries over time. Extensive theoretical and empirical evaluations demonstrate FlyPrompt's superior performance, achieving up to 11.23%, 12.43%, and 7.62% gains over state-of-the-art baselines on CIFAR-100, ImageNet-R, and CUB-200, respectively. Our source code is available at https://github.com/AnAppleCore/FlyGCL.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 46 of its 72 references that are in the map. First posted on arXiv 2 Feb 2026.

Builds on

46 of 72 references.

References · 46

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  4. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  5. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  6. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  7. 2019Similarity of Neural Network Representations Revisitedcited by 1,695
  8. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
  9. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,313
  10. 2021Learning to Prompt for Continual Learningcited by 938
  11. 2020Dark Experience for General Continual Learning: a Strong, Simple Baselinecited by 895
  12. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 891
34 more
  1. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  2. 2022DualPrompt: Complementary Prompting for Rehearsal-free Continual Learningcited by 635
  3. 2019Three scenarios for continual learningcited by 610
  4. 2021ImageNet-21K Pretraining for the Massescited by 607
  5. 2019Gradient based sample selection for online continual learningcited by 568
  6. 2017Learning multiple visual domains with residual adapterscited by 553
  7. 2022CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learningcited by 446
  8. 2021Rainbow Memory: Continual Learning with a Memory of Diverse Samplescited by 273
  9. 2022S-Prompts Learning with Pre-trained Transformers: An Occam's Razor for Domain Incremental Learningcited by 246
  10. 2018Task-Free Continual Learningcited by 219
  11. 2023SLCA: Slow Learner with Classifier Alignment for Continual Learning on a Pre-trained Modelcited by 178
  12. 2013Non-stationary Stochastic Optimizationcited by 173
  13. 2023RanPAC: Random Projections and Pre-trained Models for Continual Learningcited by 162
  14. 2023Hierarchical Decomposition of Prompt-Based Continual Learning: Rethinking Obscured Sub-optimalitycited by 121
  15. 2022Toward Next-Generation Artificial Intelligence: Catalyzing the NeuroAI Revolutioncited by 78
  16. 2022A Theoretical Study on Solving Continual Learningcited by 78
  17. 2021AFEC: Active Forgetting of Negative Transfer in Continual Learningcited by 73
  18. 2021Online Continual Learning on Class Incremental Blurry Task Configuration with Anytime Inferencecited by 66
  19. 2025SD-LoRA: Scalable Decoupled Low-Rank Adaptation for Class Incremental Learningcited by 60
  20. 2022Beyond Not-Forgetting: Continual Learning with Backward Knowledge Transfercited by 59
  21. 2022ACIL: Analytic Class-Incremental Learning with Absolute Memorization and Privacy Protectioncited by 56
  22. 2022CoSCL: Cooperation of Small Continual Learners is Stronger than a Big Onecited by 43
  23. 2024Mixture of Experts Meets Prompt-Based Continual Learningcited by 43
  24. 2023Online Class Incremental Learning on Stochastic Blurry Task Boundary via Mask and Visual Prompt Tuningcited by 35
  25. 2024GACL: Exemplar-Free Generalized Analytic Continual Learningcited by 22
  26. 2024HiDe-PET: Continual Learning via Hierarchical Decomposition of Parameter-Efficient Tuningcited by 20
  27. 2024A Comprehensive Survey of Action Quality Assessment: Method and Benchmarkcited by 20
  28. 2024Orchestrate Latent Expertise: Advancing Online Continual Learning with Multi-Level Supervision and Reverse Self-Distillationcited by 16
  29. 2024MAGR: Manifold-Aligned Graph Regularization for Continual Action Quality Assessmentcited by 13
  30. 2025Advancing Prompt-Based Methods for Replay-Independent General Continual Learningcited by 9
  31. 2025Adaptive Score Alignment Learning for Continual Perceptual Quality Assessment of 360-Degree Videos in Virtual Realitycited by 6
  32. 2025Continual Action Quality Assessment via Adaptive Manifold-Aligned Graph Regularizationcited by 4
  33. 2025Domain Generalizable Continual Learningcited by 2
  34. 2025Right Time to Learn:Promoting Generalization via Bio-inspired Spacing Effect in Knowledge Distillationcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond Routing Saturation: A Long-Horizon Class-Incremental Perspective on Expert Routing in Multimodal Continual Instruction Tuning
  2. 2026Brain-Inspired Hierarchical Modularity for General Continual Learning
  3. 2026PASs-MoE: Mitigating Misaligned Co-drift among Router and Experts via Pathway Activation Subspaces for Continual Learning
  4. 2025One-Prompt Strikes Back: Sparse Mixture of Experts for Prompt-based Continual Learning
  5. 2026SAME: Stabilized Mixture-of-Experts for Multimodal Continual Instruction Tuning
  6. 2026DIMoE-Adapters: Dynamic Expert Evolution for Continual Learning in Vision-Language Models
  7. 2026On Token's Dilemma: Dynamic MoE with Drift-Aware Token Assignment for Continual Learning of Large Vision Language Models
  8. 2026CP-MoE: Consistency-Preserving Mixture-of-Experts for Continual Learning