# Federated Action Recognition for Smart Worker Assistance Using FastPose

arXiv: https://arxiv.org/abs/2508.14113 · HTML: https://mlsanity.com/paper/2508.14113/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 26 references that are in the map.
- First posted on arXiv 18 Aug 2025.

## Builds on (most cited first, 18 of 18)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis](https://mlsanity.com/paper/1604.02808/) (2016)
- [Federated Learning for Mobile Keyboard Prediction](https://mlsanity.com/paper/1811.03604/) (2018)
- [Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translation](https://mlsanity.com/paper/2003.13830/) (2020)
- [BlazePose: On-device Real-time Body Pose tracking](https://mlsanity.com/paper/2006.10204/) (2020)
- [Skeleton-based Action Recognition via Spatial and Temporal Transformer Networks](https://mlsanity.com/paper/2008.07404/) (2020)
- [Skeleton-Based Human Action Recognition with Global Context-Aware Attention LSTM Networks](https://mlsanity.com/paper/1707.05740/) (2017)
- [A Survey on 3D Skeleton-Based Action Recognition Using Learning Method](https://mlsanity.com/paper/2002.05907/) (2020)
- [FastPose: Towards Real-time Pose Estimation and Tracking via Scale-normalized Multi-task Networks](https://mlsanity.com/paper/1908.05593/) (2019)
- [Application of federated learning in manufacturing](https://mlsanity.com/paper/2208.04664/) (2022)
- [FSAR: Federated Skeleton-based Action Recognition with Adaptive Topology Structure and Knowledge Distillation](https://mlsanity.com/paper/2306.11046/) (2023)
- [Federated Ensemble YOLOv5 -- A Better Generalized Object Detection Algorithm](https://mlsanity.com/paper/2306.17829/) (2023)
- [Enhancing Object Detection with Hybrid dataset in Manufacturing Environments: Comparing Federated Learning to Conventional Techniques](https://mlsanity.com/paper/2408.08974/) (2024)
- [Federated Object Detection for Quality Inspection in Shared Production](https://mlsanity.com/paper/2306.17645/) (2023)
- [Addressing Heterogeneity in Federated Learning: Challenges and Solutions for a Shared Production Environment](https://mlsanity.com/paper/2408.09556/) (2024)

## Built on by (most cited first, 1 of 1)

- [Unlocking Multi-Site Clinical Data: A Federated Approach to Privacy-First Child Autism Behavior Analysis](https://mlsanity.com/paper/2604.02616/) (2026)

## Abstract (arXiv)

In smart manufacturing environments, accurate and real-time recognition of worker actions is essential for productivity, safety, and human-machine collaboration. While skeleton-based human activity recognition (HAR) offers robustness to lighting, viewpoint, and background variations, most existing approaches rely on centralized datasets, which are impractical in privacy-sensitive industrial scenarios. This paper presents a federated learning (FL) framework for pose-based HAR using a custom skeletal dataset of eight industrially relevant upper-body gestures, captured from five participants and processed using a modified FastPose model. Two temporal backbones, an LSTM and a Transformer encoder, are trained and evaluated under four paradigms: centralized, local (per-client), FL with weighted federated averaging (FedAvg), and federated ensemble learning (FedEnsemble). On the global test set, the FL Transformer improves over centralized training by +12.4 percentage points, with FedEnsemble delivering a +16.3 percentage points gain. On an unseen external client, FL and FedEnsemble exceed centralized accuracy by +52.6 and +58.3 percentage points, respectively. These results demonstrate that FL not only preserves privacy but also substantially enhances cross-user generalization, establishing it as a practical solution for scalable, privacy-aware HAR in heterogeneous industrial settings.