# AdapNet: Adaptability Decomposing Encoder-Decoder Network for Weakly Supervised Action Recognition and Localization

arXiv: https://arxiv.org/abs/1911.11961 · HTML: https://mlsanity.com/paper/1911.11961/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Nov 2019.

## Built on by (most cited first, 8 of 8)

- [Deep Learning-based Action Detection in Untrimmed Videos: A Survey](https://mlsanity.com/paper/2110.00111/) (2021)
- [Joint-bone Fusion Graph Convolutional Network for Semi-supervised Skeleton Action Recognition](https://mlsanity.com/paper/2202.04075/) (2022)
- [Anchor-Constrained Viterbi for Set-Supervised Action Segmentation](https://mlsanity.com/paper/2104.02113/) (2021)
- [DDG-Net: Discriminability-Driven Graph Network for Weakly-supervised Temporal Action Localization](https://mlsanity.com/paper/2307.16415/) (2023)
- [Decomposing a Recurrent Neural Network into Modules for Enabling Reusability and Replacement](https://mlsanity.com/paper/2212.05970/) (2022)
- [Auxiliary Tasks Enhanced Dual-affinity Learning for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2403.01156/) (2024)
- [Weakly-Supervised Temporal Action Localization with Bidirectional Semantic Consistency Constraint](https://mlsanity.com/paper/2304.12616/) (2023)
- [DiffusionVMR: Diffusion Model for Joint Video Moment Retrieval and Highlight Detection](https://mlsanity.com/paper/2308.15109/) (2023)

## Abstract (arXiv)

The point process is a solid framework to model sequential data, such as videos, by exploring the underlying relevance. As a challenging problem for high-level video understanding, weakly supervised action recognition and localization in untrimmed videos has attracted intensive research attention. Knowledge transfer by leveraging the publicly available trimmed videos as external guidance is a promising attempt to make up for the coarse-grained video-level annotation and improve the generalization performance. However, unconstrained knowledge transfer may bring about irrelevant noise and jeopardize the learning model. This paper proposes a novel adaptability decomposing encoder-decoder network to transfer reliable knowledge between trimmed and untrimmed videos for action recognition and localization via bidirectional point process modeling, given only video-level annotations. By decomposing the original features into domain-adaptable and domain-specific ones based on their adaptability, trimmed-untrimmed knowledge transfer can be safely confined within a more coherent subspace. An encoder-decoder based structure is carefully designed and jointly optimized to facilitate effective action classification and temporal localization. Extensive experiments are conducted on two benchmark datasets (i.e., THUMOS14 and ActivityNet1.3), and experimental results clearly corroborate the efficacy of our method.