mlsanityData to 7 Oct 2026

Meet JEANIE: a Similarity Measure for 3D Skeleton Sequences via Temporal-Viewpoint Alignment

arXiv 2402.04599 ↗First posted 7 Feb 2024cs.CV cs.AI cs.LGcited by 10

Not on the map. Why?

Abstract (arXiv)

Video sequences exhibit significant nuisance variations (undesired effects) of speed of actions, temporal locations, and subjects' poses, leading to temporal-viewpoint misalignment when comparing two sets of frames or evaluating the similarity of two sequences. Thus, we propose Joint tEmporal and cAmera viewpoiNt alIgnmEnt (JEANIE) for sequence pairs. In particular, we focus on 3D skeleton sequences whose camera and subjects' poses can be easily manipulated in 3D. We evaluate JEANIE on skeletal Few-shot Action Recognition (FSAR), where matching well temporal blocks (temporal chunks that make up a sequence) of support-query sequence pairs (by factoring out nuisance variations) is essential due to limited samples of novel classes. Given a query sequence, we create its several views by simulating several camera locations. For a support sequence, we match it with view-simulated query sequences, as in the popular Dynamic Time Warping (DTW). Specifically, each support temporal block can be matched to the query temporal block with the same or adjacent (next) temporal index, and adjacent camera views to achieve joint local temporal-viewpoint warping. JEANIE selects the smallest distance among matching paths with different temporal-viewpoint warping patterns, an advantage over DTW which only performs temporal alignment. We also propose an unsupervised FSAR akin to clustering of sequences with JEANIE as a distance measure. JEANIE achieves state-of-the-art results on NTU-60, NTU-120, Kinetics-skeleton and UWA3D Multiview Activity II on supervised and unsupervised FSAR, and their meta-learning inspired fusion.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 91 of its 168 references that are in the map. First posted on arXiv 7 Feb 2024.

Builds on

91 of 168 references.

References · 91

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  4. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  5. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  6. 2016Matching Networks for One Shot Learningcited by 2,246
  7. 2017The Kinetics Human Action Video Datasetcited by 1,894
  8. 2019Simplifying Graph Convolutional Networkscited by 1,470
  9. 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,199
  10. 2017Learning to Compare: Relation Network for Few-Shot Learningcited by 1,139
  11. 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,061
  12. 2018Predict then Propagate: Graph Neural Networks meet Personalized PageRankcited by 987
38 more
  1. 2016NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysiscited by 683
  2. 2019NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understandingcited by 530
  3. 2017A simple yet effective baseline for 3d human pose estimationcited by 401
  4. 2016Convolutional Two-Stream Network Fusion for Video Action Recognitioncited by 384
  5. 2021Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognitioncited by 317
  6. 2018Video Action Transformer Networkcited by 316
  7. 2017Temporal Segment Networks for Action Recognition in Videoscited by 279
  8. 2017Soft-DTW: a Differentiable Loss Function for Time-Seriescited by 265
  9. 2019Actional-Structural Graph Convolutional Networks for Skeleton-based Action Recognitioncited by 228
  10. 2020Scaling Graph Neural Networks with Approximate PageRankcited by 174
  11. 2019An Attention Enhanced Graph Convolutional LSTM Network for Skeleton-Based Action Recognitioncited by 151
  12. 2019Semantics-Guided Neural Networks for Efficient Skeleton-Based Human Action Recognitioncited by 135
  13. 2019Few-Shot Video Classification via Temporal Alignmentcited by 124
  14. 2020Adversarial Feature Hallucination Networks for Few-Shot Learningcited by 110
  15. 2017View Adaptive Recurrent Neural Networks for High Performance Human Action Recognition from Skeleton Datacited by 105
  16. 2020Skeleton-based Action Recognition via Spatial and Temporal Transformer Networkscited by 98
  17. 2021Constructing Stronger and Faster Baselines for Skeleton-based Action Recognitioncited by 87
  18. 2018View Adaptive Neural Networks for High Performance Skeleton-based Human Action Recognitioncited by 84
  19. 2020Spatial Temporal Transformer Network for Skeleton-based Action Recognitioncited by 76
  20. 2019Meta-Learning of Neural Architectures for Few-Shot Learningcited by 73
  21. 2022SVFormer: Semi-supervised Video Transformer for Action Recognitioncited by 67
  22. 2023Learning Discriminative Representations for Skeleton Based Action Recognitioncited by 62
  23. 2019Linkage Based Face Clustering via Graph Convolution Networkcited by 55
  24. 2021Spatio-temporal Relation Modeling for Few-shot Action Recognitioncited by 52
  25. 2022Spectral Feature Augmentation for Graph Contrastive Learning and Beyondcited by 49
  26. 2019Synthetic Humans for Action Recognition from Unseen Viewpointscited by 46
  27. 2022STAR-Transformer: A Spatio-temporal Cross Attention Transformer for Human Action Recognitioncited by 43
  28. 2019ProtoGAN: Towards Few Shot Learning for Action Recognitioncited by 41
  29. 20233Mformer: Multi-order Multi-mode Transformer for Skeletal Action Recognitioncited by 40
  30. 2018A Generative Approach to Zero-Shot and Few-Shot Action Recognitioncited by 39
  31. 2019Spatiotemporal Pyramid Network for Video Action Recognitioncited by 37
  32. 2023MoLo: Motion-augmented Long-short Contrastive Learning for Few-shot Action Recognitioncited by 36
  33. 2021Transformers Generalize DeepSets and Can be Extended to Graphs and Hypergraphscited by 35
  34. 2022Compound Prototype Matching for Few-shot Action Recognitioncited by 35
  35. 2020Rethinking Class Relations: Absolute-relative Supervised and Unsupervised Few-shot Learningcited by 31
  36. 2022DirecFormer: A Directed Attention in Transformer Approach to Robust Action Recognitioncited by 31
  37. 2020TAFSSL: Task-Adaptive Feature Sub-Space Learning for few-shot classificationcited by 30
  38. 2020Revisiting Pose-Normalization for Fine-Grained Few-Shot Recognitioncited by 30

50 most cited of 91 All 91 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 4; 2026: 4.

10 papers.

  1. 2024Learnable Expansion of Graph Operators for Multi-Modal Feature Fusioncited by 22
  2. 2024Do Language Models Understand Time?cited by 14
  3. 2025Feature Hallucination for Self-supervised Action Recognitioncited by 3
  4. 2025Learning Time in Static Classifierscited by 2
  5. 2025Evolving Skeletons: Motion Dynamics in Action Recognitioncited by 1
  6. 2025Video Understanding by Design: How Datasets Shape Video Modelscited by 0
  7. 2026Subspace Kernel Learning on Tensor Sequencescited by 0
  8. 2026Octopus: History-Free Gradient Orthogonalization for Continual Learning in Multimodal Large Language Modelscited by 0
  9. 2026Uncertainty-DTW for Sequences and Visual Tokenscited by 0
  10. 2026STAR: Skeletal Token Alignment and Rearrangement for Interaction Recognitioncited by 0