# BHaRNet: Reliability-Aware Body-Hand Modality Expertized Networks for Fine-grained Skeleton Action Recognition

arXiv: https://arxiv.org/abs/2601.00369 · HTML: https://mlsanity.com/paper/2601.00369/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 30 references that are in the map.
- First posted on arXiv 1 Jan 2026.

## Builds on (most cited first, 16 of 16)

- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis](https://mlsanity.com/paper/1604.02808/) (2016)
- [MediaPipe: A Framework for Building Perception Pipelines](https://mlsanity.com/paper/1906.08172/) (2019)
- [Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2107.12213/) (2021)
- [Semantics-Guided Neural Networks for Efficient Skeleton-Based Human Action Recognition](https://mlsanity.com/paper/1904.01189/) (2019)
- [Cross-view Action Modeling, Learning and Recognition](https://mlsanity.com/paper/1405.2941/) (2014)
- [Uni-Sign: Toward Unified Sign Language Understanding at Scale](https://mlsanity.com/paper/2501.15187/) (2025)
- [3Mformer: Multi-order Multi-mode Transformer for Skeletal Action Recognition](https://mlsanity.com/paper/2303.14474/) (2023)
- [IGFormer: Interaction Graph Transformer for Skeleton-based Human Interaction Recognition](https://mlsanity.com/paper/2207.12100/) (2022)
- [Interactive Spatiotemporal Token Attention Network for Skeleton-based General Interactive Action Recognition](https://mlsanity.com/paper/2307.07469/) (2023)
- [Unified Keypoint-based Action Recognition Framework via Structured Keypoint Pooling](https://mlsanity.com/paper/2303.15270/) (2023)
- [On the Utility of 3D Hand Poses for Action Recognition](https://mlsanity.com/paper/2403.09805/) (2024)
- [Skeleton-Based Human Action Recognition with Noisy Labels](https://mlsanity.com/paper/2403.09975/) (2024)
- [Expressive Keypoints for Skeleton-based Action Recognition via Skeleton Transformation](https://mlsanity.com/paper/2406.18011/) (2024)
- [Expressive Forecasting of 3D Whole-body Human Motions](https://mlsanity.com/paper/2312.11972/) (2023)
- [Body-Hand Modality Expertized Networks with Cross-attention for Fine-grained Skeleton Action Recognition](https://mlsanity.com/paper/2503.14960/) (2025)

## Abstract (arXiv)

Skeleton-based human action recognition (HAR) has achieved remarkable progress with graph-based architectures. However, most existing methods remain body-centric, focusing on large-scale motions while neglecting subtle hand articulations that are crucial for fine-grained recognition. This work presents a probabilistic dual-stream framework that unifies reliability modeling and multi-modal integration, generalizing expertized learning under uncertainty across both intra-skeleton and cross-modal domains. The framework comprises three key components: (1) a calibration-free preprocessing pipeline that removes canonical-space transformations and learns directly from native coordinates; (2) a probabilistic Noisy-OR fusion that stabilizes reliability-aware dual-stream learning without requiring explicit confidence supervision; and (3) an intra- to cross-modal ensemble that couples four skeleton modalities (Joint, Bone, Joint Motion, and Bone Motion) to RGB representations, bridging structural and visual motion cues in a unified cross-modal formulation. Comprehensive evaluations across multiple benchmarks (NTU RGB+D~60/120, PKU-MMD, N-UCLA) and a newly defined hand-centric benchmark exhibit consistent improvements and robustness under noisy and heterogeneous conditions.