# SeFAR: Semi-supervised Fine-grained Action Recognition with Temporal Perturbation and Learning Stabilization

arXiv: https://arxiv.org/abs/2501.01245 · HTML: https://mlsanity.com/paper/2501.01245/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Jan 2025.

## Built on by (most cited first, 8 of 8)

- [LightGen: Efficient Image Generation through Knowledge Distillation and Direct Preference Optimization](https://mlsanity.com/paper/2503.08619/) (2025)
- [VistaDPO: Video Hierarchical Spatial-Temporal Direct Preference Optimization for Large Video Models](https://mlsanity.com/paper/2504.13122/) (2025)
- [Temporal Regularization Makes Your Video Generator Stronger](https://mlsanity.com/paper/2503.15417/) (2025)
- [FinePhys: Fine-grained Human Action Generation by Explicitly Incorporating Physical Laws for Effective Skeletal Guidance](https://mlsanity.com/paper/2505.13437/) (2025)
- [Video-based Generalized Category Discovery via Memory-Guided Consistency-Aware Contrastive Learning](https://mlsanity.com/paper/2509.06306/) (2025)
- [Learnable Motion-Focused Tokenization for Effective and Efficient Video Unsupervised Domain Adaptation](https://mlsanity.com/paper/2604.09955/) (2026)
- [LEViL: Label-Efficient Video Learning via Zero-Shot Distillation over VLM-Generated Pseudo-Label Spaces](https://mlsanity.com/paper/2606.21358/) (2026)
- [SoftRerank: Hierarchical Soft Fusion with Candidate-Label Reranking for Long-Tailed Micro-Action Recognition](https://mlsanity.com/paper/2609.08221/) (2026)

## Abstract (arXiv)

Human action understanding is crucial for the advancement of multimodal systems. While recent developments, driven by powerful large language models (LLMs), aim to be general enough to cover a wide range of categories, they often overlook the need for more specific capabilities. In this work, we address the more challenging task of Fine-grained Action Recognition (FAR), which focuses on detailed semantic labels within shorter temporal duration (e.g., "salto backward tucked with 1 turn"). Given the high costs of annotating fine-grained labels and the substantial data needed for fine-tuning LLMs, we propose to adopt semi-supervised learning (SSL). Our framework, SeFAR, incorporates several innovative designs to tackle these challenges. Specifically, to capture sufficient visual details, we construct Dual-level temporal elements as more effective representations, based on which we design a new strong augmentation strategy for the Teacher-Student learning paradigm through involving moderate temporal perturbation. Furthermore, to handle the high uncertainty within the teacher model's predictions for FAR, we propose the Adaptive Regulation to stabilize the learning process. Experiments show that SeFAR achieves state-of-the-art performance on two FAR datasets, FineGym and FineDiving, across various data scopes. It also outperforms other semi-supervised methods on two classical coarse-grained datasets, UCF101 and HMDB51. Further analysis and ablation studies validate the effectiveness of our designs. Additionally, we show that the features extracted by our SeFAR could largely promote the ability of multimodal foundation models to understand fine-grained and domain-specific semantics.