# Temporal-Viewpoint Transportation Plan for Skeletal Few-shot Action Recognition

arXiv: https://arxiv.org/abs/2210.16820 · HTML: https://mlsanity.com/paper/2210.16820/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 52 of its 97 references that are in the map.
- First posted on arXiv 30 Oct 2022.

## Builds on (most cited first, 30 of 52)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [Simplifying Graph Convolutional Networks](https://mlsanity.com/paper/1902.07153/) (2019)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [Predict then Propagate: Graph Neural Networks meet Personalized PageRank](https://mlsanity.com/paper/1810.05997/) (2018)
- [NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis](https://mlsanity.com/paper/1604.02808/) (2016)
- [NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding](https://mlsanity.com/paper/1905.04757/) (2019)
- [A simple yet effective baseline for 3d human pose estimation](https://mlsanity.com/paper/1705.03098/) (2017)
- [Convolutional Two-Stream Network Fusion for Video Action Recognition](https://mlsanity.com/paper/1604.06573/) (2016)
- [Disentangling and Unifying Graph Convolutions for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2003.14111/) (2020)
- [Temporal Segment Networks for Action Recognition in Videos](https://mlsanity.com/paper/1705.02953/) (2017)
- [Soft-DTW: a Differentiable Loss Function for Time-Series](https://mlsanity.com/paper/1703.01541/) (2017)
- [Actional-Structural Graph Convolutional Networks for Skeleton-based Action Recognition](https://mlsanity.com/paper/1904.12659/) (2019)
- [A New Representation of Skeleton Sequences for 3D Action Recognition](https://mlsanity.com/paper/1703.03492/) (2017)
- [Few-Shot Video Classification via Temporal Alignment](https://mlsanity.com/paper/1906.11415/) (2019)
- [Adversarial Feature Hallucination Networks for Few-Shot Learning](https://mlsanity.com/paper/2003.13193/) (2020)
- [View Adaptive Recurrent Neural Networks for High Performance Human Action Recognition from Skeleton Data](https://mlsanity.com/paper/1703.08274/) (2017)
- [On Learning the Geodesic Path for Incremental Learning](https://mlsanity.com/paper/2104.08572/) (2021)
- [View Adaptive Neural Networks for High Performance Skeleton-based Human Action Recognition](https://mlsanity.com/paper/1804.07453/) (2018)
- [Meta-Learning of Neural Architectures for Few-Shot Learning](https://mlsanity.com/paper/1911.11090/) (2019)
- [ProtoGAN: Towards Few Shot Learning for Action Recognition](https://mlsanity.com/paper/1909.07945/) (2019)
- [A Generative Approach to Zero-Shot and Few-Shot Action Recognition](https://mlsanity.com/paper/1801.09086/) (2018)
- [Spatiotemporal Pyramid Network for Video Action Recognition](https://mlsanity.com/paper/1903.01038/) (2019)

## Built on by (most cited first, 22 of 22)

- [Advancing Video Anomaly Detection: A Concise Review and a New Dataset](https://mlsanity.com/paper/2402.04857/) (2024)
- [3Mformer: Multi-order Multi-mode Transformer for Skeletal Action Recognition](https://mlsanity.com/paper/2303.14474/) (2023)
- [Transductive Few-shot Learning with Prototype-based Label Propagation by Iterative Graph Refinement](https://mlsanity.com/paper/2304.11598/) (2023)
- [Distilling Self-Supervised Vision Transformers for Weakly-Supervised Few-Shot Classification & Segmentation](https://mlsanity.com/paper/2307.03407/) (2023)
- [Learnable Expansion of Graph Operators for Multi-Modal Feature Fusion](https://mlsanity.com/paper/2410.01506/) (2024)
- [Do Language Models Understand Time?](https://mlsanity.com/paper/2412.13845/) (2024)
- [One-Shot Action Recognition via Multi-Scale Spatial-Temporal Skeleton Matching](https://mlsanity.com/paper/2307.07286/) (2023)
- [Flow Dynamics Correction for Action Recognition](https://mlsanity.com/paper/2310.10059/) (2023)
- [Traffic Forecasting on New Roads Using Spatial Contrastive Pre-Training (SCPT)](https://mlsanity.com/paper/2305.05237/) (2023)
- [A Comprehensive Review of Few-shot Action Recognition](https://mlsanity.com/paper/2407.14744/) (2024)
- [Meet JEANIE: a Similarity Measure for 3D Skeleton Sequences via Temporal-Viewpoint Alignment](https://mlsanity.com/paper/2402.04599/) (2024)
- [3D Skeleton-Based Action Recognition: A Review](https://mlsanity.com/paper/2506.00915/) (2025)
- [MoFM: A Large-Scale Human Motion Foundation Model](https://mlsanity.com/paper/2502.05432/) (2025)
- [SkeletonX: Data-Efficient Skeleton-based Action Recognition via Cross-sample Feature Aggregation](https://mlsanity.com/paper/2504.11749/) (2025)
- [Feature Hallucination for Self-supervised Action Recognition](https://mlsanity.com/paper/2506.20342/) (2025)
- [Exploiting Field Dependencies for Learning on Categorical Data](https://mlsanity.com/paper/2307.09321/) (2023)
- [Few-Shot Learning in Video and 3D Object Detection: A Survey](https://mlsanity.com/paper/2507.17079/) (2025)
- [Learning Time in Static Classifiers](https://mlsanity.com/paper/2511.12321/) (2025)
- [Position and Orientation-Aware One-Shot Learning for Medical Action Recognition from Signal Data](https://mlsanity.com/paper/2309.15635/) (2023)
- [Cross-Domain Human Action Recognition from Multiview Motion and Textual Descriptions](https://mlsanity.com/paper/2605.22697/) (2026)
- [Uncertainty-DTW for Sequences and Visual Tokens](https://mlsanity.com/paper/2605.25110/) (2026)
- [SkelHCC: A Hyperbolic CLIP-Driven Cache Adaptation Framework for Skeleton-based One-Shot Action Recognition](https://mlsanity.com/paper/2606.03610/) (2026)

## Abstract (arXiv)

We propose a Few-shot Learning pipeline for 3D skeleton-based action recognition by Joint tEmporal and cAmera viewpoiNt alIgnmEnt (JEANIE). To factor out misalignment between query and support sequences of 3D body joints, we propose an advanced variant of Dynamic Time Warping which jointly models each smooth path between the query and support frames to achieve simultaneously the best alignment in the temporal and simulated camera viewpoint spaces for end-to-end learning under the limited few-shot training data. Sequences are encoded with a temporal block encoder based on Simple Spectral Graph Convolution, a lightweight linear Graph Neural Network backbone. We also include a setting with a transformer. Finally, we propose a similarity-based loss which encourages the alignment of sequences of the same class while preventing the alignment of unrelated sequences. We show state-of-the-art results on NTU-60, NTU-120, Kinetics-skeleton and UWA3D Multiview Activity II.