Collaborative Adaptive Curriculum for Progressive Knowledge Distillation
Not on the map. Why?
Abstract (arXiv)
Recent advances in collaborative knowledge distillation have demonstrated cutting-edge performance for resource-constrained distributed multimedia learning scenarios. However, achieving such competitiveness requires addressing a fundamental mismatch: high-dimensional teacher knowledge complexity versus heterogeneous client learning capacities, which currently prohibits deployment in edge-based visual analytics systems. Drawing inspiration from curriculum learning principles, we introduce Federated Adaptive Progressive Distillation (FAPD), a consensus-driven framework that orchestrates adaptive knowledge transfer. FAPD hierarchically decomposes teacher features via PCA-based structuring, extracting principal components ordered by variance contribution to establish a natural visual knowledge hierarchy. Clients progressively receive knowledge of increasing complexity through dimension-adaptive projection matrices. Meanwhile, the server monitors network-wide learning stability by tracking global accuracy fluctuations across a temporal consensus window, advancing curriculum dimensionality only when collective consensus emerges. Consequently, FAPD provably adapts knowledge transfer pace while achieving superior convergence over fixed-complexity approaches. Extensive experiments on three datasets validate FAPD's effectiveness: it attains 3.64% accuracy improvement over FedAvg on CIFAR-10, demonstrates 2x faster convergence, and maintains robust performance under extreme data heterogeneity (α=0.1), outperforming baselines by over 4.5%.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 22 of its 39 references that are in the map. First posted on arXiv 19 Mar 2026.
Builds on
22 of 39 references.
References · 22
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
- 2018Federated Optimization in Heterogeneous Networkscited by 2,971
- 2021Model-Contrastive Federated Learningcited by 762
- 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 454
- 2022Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learningcited by 172
- 2022Vertical Federated Learning: Concepts, Advances and Challengescited by 104
- 2022Tackling Data Heterogeneity in Federated Learning with Class Prototypescited by 66
- 2023Federated Learning for Computationally-Constrained Heterogeneous Devices: A Surveycited by 40
- 2024Relaxed Contrastive Learning for Federated Learningcited by 32
- 2023The Best of Both Worlds: Accurate Global and Personalized Models through Federated Learning with Data-Free Hyper-Knowledge Distillationcited by 25
- 2024Networking Systems for Video Anomaly Detection: A Tutorial and Surveycited by 21
10 more
- 2025A Survey on Diffusion Models for Anomaly Detectioncited by 13
- 2025Edge-Cloud Collaborative Computing on Distributed Intelligence and Model Optimization: A Surveycited by 13
- 2025Anomaly Detection and Generation with Diffusion Models: A Surveycited by 8
- 2024DiffImpute: Tabular Data Imputation With Denoising Diffusion Probabilistic Modelcited by 6
- 2024All rivers run into the sea: Unified Modality Brain-like Emotional Central Mechanismcited by 5
- 2025Curriculum Guided Personalized Subgraph Federated Learningcited by 4
- 2025FedDifRC: Unlocking the Potential of Text-to-Image Diffusion Models in Heterogeneous Federated Learningcited by 3
- 2025A Survey on Cloud-Edge-Terminal Collaborative Intelligence in AIoT Networkscited by 3
- 2022FedCL: Federated Multi-Phase Curriculum Learning to Synchronously Correlate User Heterogeneitycited by 2
- 2024Client-supervised Federated Learning: Towards One-model-for-all Personalizationcited by 1
Built on by
Citing papers per year: 2026: 2.
2 papers.