# Drift-aware Collaborative Assistance Mixture of Experts for Heterogeneous Multistream Learning

arXiv: https://arxiv.org/abs/2508.01598 · HTML: https://mlsanity.com/paper/2508.01598/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Placed in Mixture of LoRA experts, one of 12 directions in Mixture-of-experts.
- Builds on 19 of its 41 references that are in the map.
- First posted on arXiv 3 Aug 2025.

## Placed in

- Mixture-of-experts / [Mixture of LoRA experts](https://mlsanity.com/moe/mixture-of-lora-experts/)

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning under Concept Drift: A Review](https://mlsanity.com/paper/2004.05785/) (2020)
- [The Benefit of Multitask Representation Learning](https://mlsanity.com/paper/1505.06279/) (2015)
- [River: machine learning for streaming data in Python](https://mlsanity.com/paper/2012.04740/) (2020)
- [OneNet: Enhancing Time Series Forecasting Models under Concept Drift by Online Ensembling](https://mlsanity.com/paper/2309.12659/) (2023)
- [Theory on Mixture-of-Experts in Continual Learning](https://mlsanity.com/paper/2406.16437/) (2024)
- [DDG-DA: Data Distribution Generation for Predictable Concept Drift Adaptation](https://mlsanity.com/paper/2201.04038/) (2022)
- [Active learning for data streams: a survey](https://mlsanity.com/paper/2302.08893/) (2023)
- [Edge-MoE: Memory-Efficient Multi-Task Vision Transformer Architecture with Task-level Sparsity via Mixture-of-Experts](https://mlsanity.com/paper/2305.18691/) (2023)
- [Adapting Multi-modal Large Language Model to Concept Drift From Pre-training Onwards](https://mlsanity.com/paper/2405.13459/) (2024)
- [AMF: Aggregated Mondrian Forests for Online Learning](https://mlsanity.com/paper/1906.10529/) (2019)
- [Online Boosting Adaptive Learning under Concept Drift for Multistream Classification](https://mlsanity.com/paper/2312.10841/) (2023)
- [Learning Robust Spectral Dynamics for Temporal Domain Generalization](https://mlsanity.com/paper/2505.12585/) (2025)
- [Concept Drift Detection from Multi-Class Imbalanced Data Streams](https://mlsanity.com/paper/2104.10228/) (2021)
- [Online Drift Detection with Maximum Concept Discrepancy](https://mlsanity.com/paper/2407.05375/) (2024)
- [Diverse Instances-Weighting Ensemble based on Region Drift Disagreement for Concept Drift Adaptation](https://mlsanity.com/paper/2004.05810/) (2020)
- [Early Concept Drift Detection via Prediction Uncertainty](https://mlsanity.com/paper/2412.11158/) (2024)
- [Quilt: Robust Data Segment Selection against Concept Drifts](https://mlsanity.com/paper/2312.09691/) (2023)
- [CORAL: Concept Drift Representation Learning for Co-evolving Time-series](https://mlsanity.com/paper/2501.01480/) (2025)

## Built on by (most cited first, 14 of 14)

- [TC-SSA: Token Compression via Semantic Slot Aggregation for Gigapixel Pathology Reasoning](https://mlsanity.com/paper/2603.01143/) (2026)
- [Fewer Tokens, Greater Scaling: Self-Adaptive Visual Bases for Efficient and Expansive Representation Learning](https://mlsanity.com/paper/2511.19515/) (2025)
- [XrayClaw: Cooperative-Competitive Multi-Agent Alignment for Trustworthy Chest X-ray Diagnosis](https://mlsanity.com/paper/2604.02695/) (2026)
- [Bayesian Ego-graph Inference for Networked Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2509.16606/) (2025)
- [Bandwidth-constrained Variational Message Encoding for Cooperative Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2512.11179/) (2025)
- [TPA: Next Token Probability Attribution for Detecting Hallucinations in RAG](https://mlsanity.com/paper/2512.07515/) (2025)
- [Towards Robust Endogenous Reasoning: Unifying Drift Adaptation in Non-Stationary Tuning](https://mlsanity.com/paper/2604.15705/) (2026)
- [DeepBooTS: Dual-Stream Residual Boosting for Drift-Resilient Time-Series Forecasting](https://mlsanity.com/paper/2511.06893/) (2025)
- [Inductive Subgraphs as Shortcuts: Causal Disentanglement for Heterophilic Graph Learning](https://mlsanity.com/paper/2604.19186/) (2026)
- [Autonomous Drift Learning in Data Streams: A Unified Perspective](https://mlsanity.com/paper/2605.01295/) (2026)
- [Heterogeneous Information-Bottleneck Coordination Graphs for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2605.17393/) (2026)
- [Respecting Modality Gap in Post-hoc Out-of-distribution Detection with Pre-trained Vision-Language Models](https://mlsanity.com/paper/2605.26661/) (2026)
- [FreKoo++: Learning Continuous Spectral Dynamics for Temporal Domain Generalization](https://mlsanity.com/paper/2608.22224/) (2026)
- [Task-Aware Federated Fine-Tuning for MoE-based Large Language Models](https://mlsanity.com/paper/2609.13395/) (2026)

## Abstract (arXiv)

Learning from multiple data streams in real-world scenarios is fundamentally challenging due to intrinsic heterogeneity and unpredictable concept drifts. Existing methods typically assume homogeneous streams and employ static architectures with indiscriminate knowledge fusion, limiting generalizability in complex dynamic environments. To tackle this gap, we propose CAMEL, a dynamic Collaborative Assistance Mixture of Experts Learning framework. It addresses heterogeneity by assigning each stream an independent system with a dedicated feature extractor and task-specific head. Meanwhile, a dynamic pool of specialized private experts captures stream-specific idiosyncratic patterns. Crucially, collaboration across these heterogeneous streams is enabled by a dedicated assistance expert. This expert employs a multi-head attention mechanism to distill and integrate relevant context autonomously from all other concurrent streams. It facilitates targeted knowledge transfer while inherently mitigating negative transfer from irrelevant sources. Furthermore, we propose an Autonomous Expert Tuner (AET) strategy, which dynamically manages expert lifecycles in response to drift. It instantiates new experts for emerging concepts (freezing prior ones to prevent catastrophic forgetting) and prunes obsolete ones. This expert-level plasticity provides a robust and efficient mechanism for online model capacity adaptation. Extensive experiments demonstrate CAMEL's superior generalizability across diverse multistreams and exceptional resilience against complex concept drifts.