# SL-DML: Signal Level Deep Metric Learning for Multimodal One-Shot Action Recognition

arXiv: https://arxiv.org/abs/2004.11085 · HTML: https://mlsanity.com/paper/2004.11085/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Apr 2020.

## Built on by (most cited first, 18 of 18)

- [MotionBERT: A Unified Perspective on Learning Human Motion Representations](https://mlsanity.com/paper/2210.06551/) (2022)
- [Temporal-Viewpoint Transportation Plan for Skeletal Few-shot Action Recognition](https://mlsanity.com/paper/2210.16820/) (2022)
- [Uncertainty-DTW for Time Series and Sequences](https://mlsanity.com/paper/2211.00005/) (2022)
- [Delving Deep into One-Shot Skeleton-based Action Recognition with Diverse Occlusions](https://mlsanity.com/paper/2202.11423/) (2022)
- [One-Shot Action Recognition via Multi-Scale Spatial-Temporal Skeleton Matching](https://mlsanity.com/paper/2307.07286/) (2023)
- [CrossGLG: LLM Guides One-shot Skeleton-based 3D Action Recognition in a Cross-level Manner](https://mlsanity.com/paper/2403.10082/) (2024)
- [3D Skeleton-based Few-shot Action Recognition with JEANIE is not so Na"ive](https://mlsanity.com/paper/2112.12668/) (2021)
- [Meet JEANIE: a Similarity Measure for 3D Skeleton Sequences via Temporal-Viewpoint Alignment](https://mlsanity.com/paper/2402.04599/) (2024)
- [Adaptive Local-Component-aware Graph Convolutional Network for One-shot Skeleton-based Action Recognition](https://mlsanity.com/paper/2209.10073/) (2022)
- [MoFM: A Large-Scale Human Motion Foundation Model](https://mlsanity.com/paper/2502.05432/) (2025)
- [Joint Self-Supervised and Supervised Contrastive Learning for Multimodal MRI Data: Towards Predicting Abnormal Neurodevelopment](https://mlsanity.com/paper/2312.15064/) (2023)
- [SkeletonX: Data-Efficient Skeleton-based Action Recognition via Cross-sample Feature Aggregation](https://mlsanity.com/paper/2504.11749/) (2025)
- [Affect-DML: Context-Aware One-Shot Recognition of Human Affect using Deep Metric Learning](https://mlsanity.com/paper/2111.15271/) (2021)
- [Part-aware Prototypical Graph Network for One-shot Skeleton-based Action Recognition](https://mlsanity.com/paper/2208.09150/) (2022)
- [Person Segmentation and Action Classification for Multi-Channel Hemisphere Field of View LiDAR Sensors](https://mlsanity.com/paper/2411.11151/) (2024)
- [Skarimva: Skeleton-based Action Recognition is a Multi-view Application](https://mlsanity.com/paper/2602.23231/) (2026)
- [SkelHCC: A Hyperbolic CLIP-Driven Cache Adaptation Framework for Skeleton-based One-Shot Action Recognition](https://mlsanity.com/paper/2606.03610/) (2026)
- [Fine-grained Human Motion Understanding with Language Models](https://mlsanity.com/paper/2606.20888/) (2026)

## Abstract (arXiv)

Recognizing an activity with a single reference sample using metric learning approaches is a promising research field. The majority of few-shot methods focus on object recognition or face-identification. We propose a metric learning approach to reduce the action recognition problem to a nearest neighbor search in embedding space. We encode signals into images and extract features using a deep residual CNN. Using triplet loss, we learn a feature embedding. The resulting encoder transforms features into an embedding space in which closer distances encode similar actions while higher distances encode different actions. Our approach is based on a signal level formulation and remains flexible across a variety of modalities. It further outperforms the baseline on the large scale NTU RGB+D 120 dataset for the One-Shot action recognition protocol by 5.6%. With just 60% of the training data, our approach still outperforms the baseline approach by 3.7%. With 40% of the training data, our approach performs comparably well to the second follow up. Further, we show that our approach generalizes well in experiments on the UTD-MHAD dataset for inertial, skeleton and fused data and the Simitate dataset for motion capturing data. Furthermore, our inter-joint and inter-sensor experiments suggest good capabilities on previously unseen setups.