FlyPrompt: Brain-Inspired Random-Expanded Routing with Temporal-Ensemble Experts for General Continual Learning
Not on the map. Why?
Abstract (arXiv)
General continual learning (GCL) challenges intelligent systems to learn from single-pass, non-stationary data streams without clear task boundaries. While recent advances in continual parameter-efficient tuning (PET) of pretrained models show promise, they typically rely on multiple training epochs and explicit task cues, limiting their effectiveness in GCL scenarios. Moreover, existing methods often lack targeted design and fail to address two fundamental challenges in continual PET: how to allocate expert parameters to evolving data distributions, and how to improve their representational capacity under limited supervision. Inspired by the fruit fly's hierarchical memory system characterized by sparse expansion and modular ensembles, we propose FlyPrompt, a brain-inspired framework that decomposes GCL into two subproblems: expert routing and expert competence improvement. FlyPrompt introduces a randomly expanded analytic router for instance-level expert activation and a temporal ensemble of output heads to dynamically adapt decision boundaries over time. Extensive theoretical and empirical evaluations demonstrate FlyPrompt's superior performance, achieving up to 11.23%, 12.43%, and 7.62% gains over state-of-the-art baselines on CIFAR-100, ImageNet-R, and CUB-200, respectively. Our source code is available at https://github.com/AnAppleCore/FlyGCL.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 46 of its 72 references that are in the map. First posted on arXiv 2 Feb 2026.
Builds on
46 of 72 references.
References · 46
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
- 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
- 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
- 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
- 2019Similarity of Neural Network Representations Revisitedcited by 1,695
- 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
- 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,313
- 2021Learning to Prompt for Continual Learningcited by 938
- 2020Dark Experience for General Continual Learning: a Strong, Simple Baselinecited by 895
- 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 891
34 more
- 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
- 2022DualPrompt: Complementary Prompting for Rehearsal-free Continual Learningcited by 635
- 2019Three scenarios for continual learningcited by 610
- 2021ImageNet-21K Pretraining for the Massescited by 607
- 2019Gradient based sample selection for online continual learningcited by 568
- 2017Learning multiple visual domains with residual adapterscited by 553
- 2022CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learningcited by 446
- 2021Rainbow Memory: Continual Learning with a Memory of Diverse Samplescited by 273
- 2022S-Prompts Learning with Pre-trained Transformers: An Occam's Razor for Domain Incremental Learningcited by 246
- 2018Task-Free Continual Learningcited by 219
- 2023SLCA: Slow Learner with Classifier Alignment for Continual Learning on a Pre-trained Modelcited by 178
- 2013Non-stationary Stochastic Optimizationcited by 173
- 2023RanPAC: Random Projections and Pre-trained Models for Continual Learningcited by 162
- 2023Hierarchical Decomposition of Prompt-Based Continual Learning: Rethinking Obscured Sub-optimalitycited by 121
- 2022Toward Next-Generation Artificial Intelligence: Catalyzing the NeuroAI Revolutioncited by 78
- 2022A Theoretical Study on Solving Continual Learningcited by 78
- 2021AFEC: Active Forgetting of Negative Transfer in Continual Learningcited by 73
- 2021Online Continual Learning on Class Incremental Blurry Task Configuration with Anytime Inferencecited by 66
- 2025SD-LoRA: Scalable Decoupled Low-Rank Adaptation for Class Incremental Learningcited by 60
- 2022Beyond Not-Forgetting: Continual Learning with Backward Knowledge Transfercited by 59
- 2022ACIL: Analytic Class-Incremental Learning with Absolute Memorization and Privacy Protectioncited by 56
- 2022CoSCL: Cooperation of Small Continual Learners is Stronger than a Big Onecited by 43
- 2024Mixture of Experts Meets Prompt-Based Continual Learningcited by 43
- 2023Online Class Incremental Learning on Stochastic Blurry Task Boundary via Mask and Visual Prompt Tuningcited by 35
- 2024GACL: Exemplar-Free Generalized Analytic Continual Learningcited by 22
- 2024HiDe-PET: Continual Learning via Hierarchical Decomposition of Parameter-Efficient Tuningcited by 20
- 2024A Comprehensive Survey of Action Quality Assessment: Method and Benchmarkcited by 20
- 2024Orchestrate Latent Expertise: Advancing Online Continual Learning with Multi-Level Supervision and Reverse Self-Distillationcited by 16
- 2024MAGR: Manifold-Aligned Graph Regularization for Continual Action Quality Assessmentcited by 13
- 2025Advancing Prompt-Based Methods for Replay-Independent General Continual Learningcited by 9
- 2025Adaptive Score Alignment Learning for Continual Perceptual Quality Assessment of 360-Degree Videos in Virtual Realitycited by 6
- 2025Continual Action Quality Assessment via Adaptive Manifold-Aligned Graph Regularizationcited by 4
- 2025Domain Generalizable Continual Learningcited by 2
- 2025Right Time to Learn:Promoting Generalization via Bio-inspired Spacing Effect in Knowledge Distillationcited by 1
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026Beyond Routing Saturation: A Long-Horizon Class-Incremental Perspective on Expert Routing in Multimodal Continual Instruction Tuning
- 2026Brain-Inspired Hierarchical Modularity for General Continual Learning
- 2026PASs-MoE: Mitigating Misaligned Co-drift among Router and Experts via Pathway Activation Subspaces for Continual Learning
- 2025One-Prompt Strikes Back: Sparse Mixture of Experts for Prompt-based Continual Learning
- 2026SAME: Stabilized Mixture-of-Experts for Multimodal Continual Instruction Tuning
- 2026DIMoE-Adapters: Dynamic Expert Evolution for Continual Learning in Vision-Language Models
- 2026On Token's Dilemma: Dynamic MoE with Drift-Aware Token Assignment for Continual Learning of Large Vision Language Models
- 2026CP-MoE: Consistency-Preserving Mixture-of-Experts for Continual Learning