# An Efficient Federated Distillation Learning System for Multi-task Time Series Classification

arXiv: https://arxiv.org/abs/2201.00011 · HTML: https://mlsanity.com/paper/2201.00011/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 66 references that are in the map.
- First posted on arXiv 30 Dec 2021.

## Builds on (most cited first, 28 of 28)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [Model-Contrastive Federated Learning](https://mlsanity.com/paper/2103.16257/) (2021)
- [Deep learning for time series classification: a review](https://mlsanity.com/paper/1809.04356/) (2018)
- [The UCR Time Series Archive](https://mlsanity.com/paper/1810.07758/) (2018)
- [Robust and Communication-Efficient Federated Learning from Non-IID Data](https://mlsanity.com/paper/1903.02891/) (2019)
- [Federated Machine Learning: Concept and Applications](https://mlsanity.com/paper/1902.04885/) (2019)
- [InceptionTime: Finding AlexNet for Time Series Classification](https://mlsanity.com/paper/1909.04939/) (2019)
- [Multi-Task Learning with Deep Neural Networks: A Survey](https://mlsanity.com/paper/2009.09796/) (2020)
- [A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection](https://mlsanity.com/paper/1907.09693/) (2019)
- [Time Series Classification from Scratch with Deep Neural Networks: A Strong Baseline](https://mlsanity.com/paper/1611.06455/) (2016)
- [A Comprehensive Survey on Graph Anomaly Detection with Deep Learning](https://mlsanity.com/paper/2106.07178/) (2021)
- [MINIROCKET: A Very Fast (Almost) Deterministic Transform for Time Series Classification](https://mlsanity.com/paper/2012.08791/) (2020)
- [Distillation-Based Semi-Supervised Federated Learning for Communication-Efficient Collaborative Training with Non-IID Private Data](https://mlsanity.com/paper/2008.06180/) (2020)
- [Multivariate LSTM-FCNs for Time Series Classification](https://mlsanity.com/paper/1801.04503/) (2018)
- [Communication-Efficient Federated Deep Learning with Asynchronous Model Update and Temporally Weighted Aggregation](https://mlsanity.com/paper/1903.07424/) (2019)
- [A Time Series Forest for Classification and Feature Extraction](https://mlsanity.com/paper/1302.2277/) (2013)
- [Provably Secure Federated Learning against Malicious Clients](https://mlsanity.com/paper/2102.01854/) (2021)
- [Federated Knowledge Distillation](https://mlsanity.com/paper/2011.02367/) (2020)
- [FLOP: Federated Learning on Medical Datasets using Partial Networks](https://mlsanity.com/paper/2102.05218/) (2021)
- [Game of Gradients: Mitigating Irrelevant Clients in Federated Learning](https://mlsanity.com/paper/2110.12257/) (2021)
- [PFA: Privacy-preserving Federated Adaptation for Effective Model Personalization](https://mlsanity.com/paper/2103.01548/) (2021)
- [From BOP to BOSS and Beyond: Time Series Classification with Dictionary Based Classifiers](https://mlsanity.com/paper/1809.06751/) (2018)
- [Learnable Dynamic Temporal Pooling for Time Series Classification](https://mlsanity.com/paper/2104.02577/) (2021)
- [RTFN: A Robust Temporal Feature Network for Time Series Classification](https://mlsanity.com/paper/2011.11829/) (2020)
- [Federated Transfer Learning with Dynamic Gradient Aggregation](https://mlsanity.com/paper/2008.02452/) (2020)
- [XEM: An Explainable-by-Design Ensemble Method for Multivariate Time Series Classification](https://mlsanity.com/paper/2005.03645/) (2020)

## Built on by (most cited first, 14 of 14)

- [Knowledge Distillation in Federated Learning: a Survey on Long Lasting Challenges and New Solutions](https://mlsanity.com/paper/2406.10861/) (2024)
- [Improving Local Training in Federated Learning via Temperature Scaling](https://mlsanity.com/paper/2401.09986/) (2024)
- [Seamless Detection: Unifying Salient Object Detection and Camouflaged Object Detection](https://mlsanity.com/paper/2412.16840/) (2024)
- [Learning to Reason: Temporal Saliency Distillation for Interpretable Knowledge Transfer](https://mlsanity.com/paper/2601.04263/) (2026)
- [Feature construction using explanations of individual predictions](https://mlsanity.com/paper/2301.09631/) (2023)
- [FedST: Secure Federated Shapelet Transformation for Time Series Classification](https://mlsanity.com/paper/2302.10631/) (2023)
- [An Automated Machine Learning Approach for Detecting Anomalous Peak Patterns in Time Series Data from a Research Watershed in the Northeastern United States Critical Zone](https://mlsanity.com/paper/2309.07992/) (2023)
- [Neural Message Passing for Objective-Based Uncertainty Quantification and Optimal Experimental Design](https://mlsanity.com/paper/2203.07120/) (2022)
- [Fuzzy Knowledge Distillation from High-Order TSK to Low-Order TSK](https://mlsanity.com/paper/2302.08038/) (2023)
- [A Hybrid Federated Kernel Regularized Least Squares Algorithm](https://mlsanity.com/paper/2407.17228/) (2024)
- [ColNet: Collaborative Optimization in Decentralized Federated Multi-task Learning Systems](https://mlsanity.com/paper/2501.10347/) (2025)
- [Provably Near-Optimal Federated Ensemble Distillation with Negligible Overhead](https://mlsanity.com/paper/2502.06349/) (2025)
- [MemKD: Memory-Discrepancy Knowledge Distillation for Efficient Time Series Classification](https://mlsanity.com/paper/2601.04264/) (2026)
- [Probabilistic Federated Learning on Uncertain and Heterogeneous Data with Model Personalization](https://mlsanity.com/paper/2603.18083/) (2026)

## Abstract (arXiv)

This paper proposes an efficient federated distillation learning system (EFDLS) for multi-task time series classification (TSC). EFDLS consists of a central server and multiple mobile users, where different users may run different TSC tasks. EFDLS has two novel components, namely a feature-based student-teacher (FBST) framework and a distance-based weights matching (DBWM) scheme. Within each user, the FBST framework transfers knowledge from its teacher's hidden layers to its student's hidden layers via knowledge distillation, with the teacher and student having identical network structure. For each connected user, its student model's hidden layers' weights are uploaded to the EFDLS server periodically. The DBWM scheme is deployed on the server, with the least square distance used to measure the similarity between the weights of two given models. This scheme finds a partner for each connected user such that the user's and its partner's weights are the closest among all the weights uploaded. The server exchanges and sends back the user's and its partner's weights to these two users which then load the received weights to their teachers' hidden layers. Experimental results show that the proposed EFDLS achieves excellent performance on a set of selected UCR2018 datasets regarding top-1 accuracy.