mlsanityData to 14 Oct 2026

Frequency-Semantic Enhanced Variational Autoencoder for Zero-Shot Skeleton-based Action Recognition

arXiv 2506.22179 ↗First posted 27 Jun 2025cs.CV cs.AIcited by 6

Not on the map. Why?

Abstract (arXiv)

Zero-shot skeleton-based action recognition aims to develop models capable of identifying actions beyond the categories encountered during training. Previous approaches have primarily focused on aligning visual and semantic representations but often overlooked the importance of fine-grained action patterns in the semantic space (e.g., the hand movements in drinking water and brushing teeth). To address these limitations, we propose a Frequency-Semantic Enhanced Variational Autoencoder (FS-VAE) to explore the skeleton semantic representation learning with frequency decomposition. FS-VAE consists of three key components: 1) a frequency-based enhancement module with high- and low-frequency adjustments to enrich the skeletal semantics learning and improve the robustness of zero-shot action recognition; 2) a semantic-based action description with multilevel alignment to capture both local details and global correspondence, effectively bridging the semantic gap and compensating for the inherent loss of information in skeleton sequences; 3) a calibrated cross-alignment loss that enables valid skeleton-text pairs to counterbalance ambiguous ones, mitigating discrepancies and ambiguities in skeleton and text features, thereby ensuring robust alignment. Evaluations on the benchmarks demonstrate the effectiveness of our approach, validating that frequency-enhanced semantic features enable robust differentiation of visually and semantically similar action clusters, improving zero-shot action recognition.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 32 of its 51 references that are in the map. First posted on arXiv 27 Jun 2025.

Builds on

32 of 51 references.

References · 32

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,849
  4. 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,209
  5. 2017In Defense of the Triplet Loss for Person Re-Identificationcited by 786
  6. 2016NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysiscited by 686
  7. 2019NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understandingcited by 533
  8. 2014Deep metric learning using Triplet networkcited by 422
  9. 2021Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognitioncited by 322
  10. 20213D Human Pose Estimation with Spatial and Temporal Transformerscited by 272
  11. 2019Learning Trajectory Dependencies for Human Motion Predictioncited by 246
  12. 2020Dynamic Multiscale Graph Neural Networks for 3D Skeleton-Based Human Motion Predictioncited by 132
20 more
  1. 2017A New Representation of Skeleton Sequences for 3D Action Recognitioncited by 126
  2. 2018Deep Metric Learning with Hierarchical Triplet Losscited by 112
  3. 2017PKU-MMD: A Large Scale Benchmark for Continuous Multi-Modal Human Action Understandingcited by 101
  4. 2019Fine-Grained Action Retrieval Through Multiple Parts-of-Speech Embeddingscited by 101
  5. 2023PoseFormerV2: Exploring Frequency Domain for Efficient and Robust 3D Human Pose Estimationcited by 92
  6. 2021Multi-Person 3D Motion Prediction with Multi-Range Transformerscited by 63
  7. 2017Learning Robust Visual-Semantic Embeddingscited by 56
  8. 2019On Symmetric Losses for Learning from Corrupted Labelscited by 53
  9. 2015Learning Local Image Descriptors with Deep Siamese and Triplet Convolutional Networks by Minimising Global Loss Functionscited by 33
  10. 2022Skeleton-Parted Graph Scattering Networks for 3D Human Motion Predictioncited by 29
  11. 2019A Theoretically Sound Upper Bound on the Triplet Loss for Improving the Efficiency of Deep Distance Metric Learningcited by 25
  12. 2021Syntactically Guided Generative Embeddings for Zero-Shot Skeleton Action Recognitioncited by 24
  13. 2023Zero-shot Skeleton-based Action Recognition via Mutual Information Estimation and Maximizationcited by 24
  14. 2024Part-aware Unified Representation of Language and Skeleton for Zero-shot Action Recognitioncited by 23
  15. 2024Fine-Grained Side Information Guided Dual-Prompts for Zero-Shot Skeleton Action Recognitioncited by 19
  16. 2024SA-DVAE: Improving Zero-Shot Skeleton-Based Action Recognition by Disentangled Variational Autoencoderscited by 19
  17. 2024Frequency Guidance Matters: Skeletal Action Recognition by Frequency-Aware Mixed Transformercited by 15
  18. 2022FeDXL: Provable Federated Learning for Deep X-Risk Optimizationcited by 11
  19. 2023Multi-Semantic Fusion Model for Generalized Zero-Shot Skeleton-Based Action Recognitioncited by 9
  20. 2024Wavelet-Decoupling Contrastive Enhancement Network for Fine-Grained Skeleton-Based Action Recognitioncited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 5.

6 papers.

  1. 2025Learning by Neighbor-Aware Semantics, Deciding by Open-form Flows: Towards Robust Zero-Shot Skeleton Action Recognitioncited by 3
  2. 2026SkeletonContext: Skeleton-side Context Prompt Learning for Zero-Shot Skeleton-based Action Recognitioncited by 3
  3. 2026Frequency-Enhanced Diffusion Models: Curriculum-Guided Semantic Alignment for Zero-Shot Skeleton Action Recognitioncited by 2
  4. 2026Toward Universal Skeleton-Based Action Recognition across Heterogeneous Skeletons and Open Vocabulariescited by 0
  5. 2026PoseBridge: Bridging the Skeletonization Gap for Zero-Shot Skeleton-Based Action Recognitioncited by 0
  6. 2026Visual Anchoring in Diffusion: Multimodal Zero-Shot Skeleton Action Recognitioncited by 0