# Learning by Neighbor-Aware Semantics, Deciding by Open-form Flows: Towards Robust Zero-Shot Skeleton Action Recognition

arXiv: https://arxiv.org/abs/2511.09388 · HTML: https://mlsanity.com/paper/2511.09388/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 49 references that are in the map.
- First posted on arXiv 12 Nov 2025.

## Builds on (most cited first, 30 of 36)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Building Normalizing Flows with Stochastic Interpolants](https://mlsanity.com/paper/2209.15571/) (2022)
- [NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis](https://mlsanity.com/paper/1604.02808/) (2016)
- [Movie Gen: A Cast of Media Foundation Models](https://mlsanity.com/paper/2410.13720/) (2024)
- [NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding](https://mlsanity.com/paper/1905.04757/) (2019)
- [Flow Matching Guide and Code](https://mlsanity.com/paper/2412.06264/) (2024)
- [A Survey on Vision-Language-Action Models for Embodied AI](https://mlsanity.com/paper/2405.14093/) (2024)
- [Audiobox: Unified Audio Generation with Natural Language Prompts](https://mlsanity.com/paper/2312.15821/) (2023)
- [PKU-MMD: A Large Scale Benchmark for Continuous Multi-Modal Human Action Understanding](https://mlsanity.com/paper/1703.07475/) (2017)
- [Fine-Grained Action Retrieval Through Multiple Parts-of-Speech Embeddings](https://mlsanity.com/paper/1908.03477/) (2019)
- [Learning Robust Visual-Semantic Embeddings](https://mlsanity.com/paper/1703.05908/) (2017)
- [Motion Flow Matching for Human Motion Synthesis and Editing](https://mlsanity.com/paper/2312.08895/) (2023)
- [FlowTok: Flowing Seamlessly Across Text and Image Tokens](https://mlsanity.com/paper/2503.10772/) (2025)
- [Video Pose Distillation for Few-Shot, Fine-Grained Sports Action Recognition](https://mlsanity.com/paper/2109.01305/) (2021)
- [Contrastive Flow Matching](https://mlsanity.com/paper/2506.05350/) (2025)
- [Connect, Collapse, Corrupt: Learning Cross-Modal Tasks with Uni-Modal Data](https://mlsanity.com/paper/2401.08567/) (2024)
- [Flowing from Words to Pixels: A Noise-Free Framework for Cross-Modality Evolution](https://mlsanity.com/paper/2412.15213/) (2024)
- [VITA: Vision-to-Action Flow Matching Policy](https://mlsanity.com/paper/2507.13231/) (2025)
- [Zero-shot Skeleton-based Action Recognition via Mutual Information Estimation and Maximization](https://mlsanity.com/paper/2308.03950/) (2023)
- [Part-aware Unified Representation of Language and Skeleton for Zero-shot Action Recognition](https://mlsanity.com/paper/2406.13327/) (2024)
- [Prompt-Guided Zero-Shot Anomaly Action Recognition using Pretrained Deep Skeleton Features](https://mlsanity.com/paper/2303.15167/) (2023)
- [Towards Universal Soccer Video Understanding](https://mlsanity.com/paper/2412.01820/) (2024)
- [Skeleton based Zero Shot Action Recognition in Joint Pose-Language Semantic Space](https://mlsanity.com/paper/1911.11344/) (2019)
- [Fine-Grained Side Information Guided Dual-Prompts for Zero-Shot Skeleton Action Recognition](https://mlsanity.com/paper/2404.07487/) (2024)
- [SA-DVAE: Improving Zero-Shot Skeleton-Based Action Recognition by Disentangled Variational Autoencoders](https://mlsanity.com/paper/2407.13460/) (2024)
- [Skeletal Video Anomaly Detection using Deep Learning: Survey, Challenges and Future Directions](https://mlsanity.com/paper/2301.00114/) (2022)

## Built on by (most cited first, 3 of 3)

- [PoseBridge: Bridging the Skeletonization Gap for Zero-Shot Skeleton-Based Action Recognition](https://mlsanity.com/paper/2605.11497/) (2026)
- [GenPrior: Unleashing Text-to-Motion Generative Priors for Zero-Shot Skeleton-based Action Recognition](https://mlsanity.com/paper/2608.02236/) (2026)
- [Visual Anchoring in Diffusion: Multimodal Zero-Shot Skeleton Action Recognition](https://mlsanity.com/paper/2608.04623/) (2026)

## Abstract (arXiv)

Recognizing unseen skeleton action categories remains highly challenging due to the absence of corresponding skeletal priors. Existing approaches generally follow an ``align-then-classify'' paradigm but face two fundamental issues, textiti.e., (i) fragile point-to-point alignment arising from imperfect semantics, and (ii) rigid classifiers restricted by static decision boundaries and coarse-grained anchors. To address these issues, we propose a novel method for zero-shot skeleton action recognition, termed textttFlora, which builds upon FlexibLe neighbOr-aware semantic attunement and open-form distRibution-aware flow clAssifier. Specifically, we flexibly attune textual semantics by incorporating neighboring inter-class contextual cues to form direction-aware regional semantics, coupled with a cross-modal geometric consistency objective that ensures stable and robust point-to-region alignment. Furthermore, we employ noise-free flow matching to bridge the modality distribution gap between semantic and skeleton latent embeddings, while a condition-free contrastive regularization enhances discriminability, leading to a distribution-aware classifier with fine-grained decision boundaries achieved through token-level velocity predictions. Extensive experiments on three benchmark datasets validate the effectiveness of our method, showing particularly impressive performance even when trained with only 10% of the seen data. Code is available at https://github.com/cseeyangchen/Flora.