mlsanityData to 14 Oct 2026

Ensemble Distribution Distillation for Self-Supervised Human Activity Recognition

arXiv 2509.08225 ↗First posted 10 Sep 2025cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Human Activity Recognition (HAR) has seen significant advancements with the adoption of deep learning techniques, yet challenges remain in terms of data requirements, reliability and robustness. This paper explores a novel application of Ensemble Distribution Distillation (EDD) within a self-supervised learning framework for HAR aimed at overcoming these challenges. By leveraging unlabeled data and a partially supervised training strategy, our approach yields an increase in predictive accuracy, robust estimates of uncertainty, and substantial increases in robustness against adversarial perturbation; thereby significantly improving reliability in real-world scenarios without increasing computational complexity at inference. We demonstrate this with an evaluation on several publicly available datasets. The contributions of this work include the development of a self-supervised EDD framework, an innovative data augmentation technique designed for HAR, and empirical validation of the proposed method's effectiveness in increasing robustness and reliability.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 37 of its 59 references that are in the map. First posted on arXiv 10 Sep 2025.

Builds on

37 of 59 references.

References · 37

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  3. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  4. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  5. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  6. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  7. 2020Knowledge Distillation: A Surveycited by 1,603
  8. 2016Variational Inference: A Review for Statisticianscited by 1,472
  9. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,287
  10. 2016Adversarial Machine Learning at Scalecited by 936
  11. 2019Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methodscited by 913
  12. 2020A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challengescited by 819
25 more
  1. 2018Predictive Uncertainty Estimation via Prior Networkscited by 557
  2. 2015Weight Uncertainty in Neural Networkscited by 554
  3. 2019Using Self-Supervised Learning Can Improve Model Robustness and Uncertaintycited by 514
  4. 2021Recent Advances in Adversarial Training for Adversarial Robustnesscited by 279
  5. 2015Bayesian Convolutional Neural Networks with Bernoulli Approximate Variational Inferencecited by 159
  6. 2019Ensemble Distribution Distillationcited by 158
  7. 2018Large scale distributed neural network training through online distillationcited by 146
  8. 2017Deep Learning for Sensor-based Activity Recognition: A Surveycited by 133
  9. 2016Crafting Adversarial Input Sequences for Recurrent Neural Networkscited by 103
  10. 2019Multi-task Self-Supervised Learning for Human Activity Detectioncited by 102
  11. 2016Deep, Convolutional, and Recurrent Models for Human Activity Recognition using Wearablescited by 94
  12. 2019Towards Unsupervised Image Captioning with Shared Multimodal Embeddingscited by 54
  13. 2019A Comprehensive guide to Bayesian Convolutional Neural Network with Variational Inferencecited by 51
  14. 2018Improving the Generalization of Adversarial Training with Domain Adaptationcited by 48
  15. 2018Protecting Sensory Data against Sensitive Inferencescited by 45
  16. 2018On the Geometry of Adversarial Examplescited by 43
  17. 2019Contextual Imagined Goals for Self-Supervised Robotic Learningcited by 40
  18. 2019Deep Sub-Ensembles for Fast Uncertainty Estimation in Image Classificationcited by 36
  19. 2020A general framework for ensemble distribution distillationcited by 16
  20. 2019On the Validity of Bayesian Neural Networks for Uncertainty Estimationcited by 14
  21. 2018Safe end-to-end imitation learning for model predictive controlcited by 12
  22. 2020Evaluating Uncertainty Estimation Methods on 3D Semantic Segmentation of Point Cloudscited by 3
  23. 2022Distributional Actor-Critic Ensemble for Uncertainty-Aware Continuous Controlcited by 3
  24. 2022Strength-Adaptive Adversarial Trainingcited by 3
  25. 2019On Norm-Agnostic Robustness of Adversarial Trainingcited by 1

Built on by

Not cited yet.