# Visual Data Synthesis via GAN for Zero-Shot Video Classification

arXiv: https://arxiv.org/abs/1804.10073 · HTML: https://mlsanity.com/paper/1804.10073/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Apr 2018.

## Built on by (most cited first, 10 of 10)

- [Domain Impression: A Source Data Free Domain Adaptation Method](https://mlsanity.com/paper/2102.09003/) (2021)
- [Multimodal Open-Vocabulary Video Classification via Pre-Trained Vision and Language Models](https://mlsanity.com/paper/2207.07646/) (2022)
- [Cross-modal Representation Learning for Zero-shot Action Recognition](https://mlsanity.com/paper/2205.01657/) (2022)
- [Alignment-Uniformity aware Representation Learning for Zero-shot Video Classification](https://mlsanity.com/paper/2203.15381/) (2022)
- [Entropy-Based Uncertainty Calibration for Generalized Zero-Shot Learning](https://mlsanity.com/paper/2101.03292/) (2021)
- [Orthogonal Temporal Interpolation for Zero-Shot Video Recognition](https://mlsanity.com/paper/2308.06897/) (2023)
- [Few Shot Activity Recognition Using Variational Inference](https://mlsanity.com/paper/2108.08990/) (2021)
- [GAN for Vision, KG for Relation: a Two-stage Deep Network for Zero-shot Action Recognition](https://mlsanity.com/paper/2105.11789/) (2021)
- [Review of Zero-Shot and Few-Shot AI Algorithms in The Medical Domain](https://mlsanity.com/paper/2406.16143/) (2024)
- [InstaIndoor and Multi-modal Deep Learning for Indoor Scene Recognition](https://mlsanity.com/paper/2112.12409/) (2021)

## Abstract (arXiv)

Zero-Shot Learning (ZSL) in video classification is a promising research direction, which aims to tackle the challenge from explosive growth of video categories. Most existing methods exploit seen-to-unseen correlation via learning a projection between visual and semantic spaces. However, such projection-based paradigms cannot fully utilize the discriminative information implied in data distribution, and commonly suffer from the information degradation issue caused by "heterogeneity gap". In this paper, we propose a visual data synthesis framework via GAN to address these problems. Specifically, both semantic knowledge and visual distribution are leveraged to synthesize video feature of unseen categories, and ZSL can be turned into typical supervised problem with the synthetic features. First, we propose multi-level semantic inference to boost video feature synthesis, which captures the discriminative information implied in joint visual-semantic distribution via feature-level and label-level semantic inference. Second, we propose Matching-aware Mutual Information Correlation to overcome information degradation issue, which captures seen-to-unseen correlation in matched and mismatched visual-semantic pairs by mutual information, providing the zero-shot synthesis procedure with robust guidance signals. Experimental results on four video datasets demonstrate that our approach can improve the zero-shot video classification performance significantly.