# Learning Robust Visual-Semantic Embeddings

arXiv: https://arxiv.org/abs/1703.05908 · HTML: https://mlsanity.com/paper/1703.05908/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Mar 2017.

## Built on by (most cited first, 30 of 56)

- [Using Shape to Categorize: Low-Shot Learning with an Explicit Shape Bias](https://mlsanity.com/paper/2101.07296/) (2021)
- [Decoding Visual Neural Representations by Multimodal Learning of Brain-Visual-Linguistic Features](https://mlsanity.com/paper/2210.06756/) (2022)
- [HSVA: Hierarchical Semantic-Visual Adaptation for Zero-Shot Learning](https://mlsanity.com/paper/2109.15163/) (2021)
- [MSDN: Mutually Semantic Distillation Network for Zero-Shot Learning](https://mlsanity.com/paper/2203.03137/) (2022)
- [All Labels Are Not Created Equal: Enhancing Semi-supervision via Label Grouping and Co-training](https://mlsanity.com/paper/2104.05248/) (2021)
- [TransZero++: Cross Attribute-Guided Transformer for Zero-Shot Learning](https://mlsanity.com/paper/2112.08643/) (2021)
- [Syntactically Guided Generative Embeddings for Zero-Shot Skeleton Action Recognition](https://mlsanity.com/paper/2101.11530/) (2021)
- [Zero-shot Skeleton-based Action Recognition via Mutual Information Estimation and Maximization](https://mlsanity.com/paper/2308.03950/) (2023)
- [Part-aware Unified Representation of Language and Skeleton for Zero-shot Action Recognition](https://mlsanity.com/paper/2406.13327/) (2024)
- [ArtFormer: Controllable Generation of Diverse 3D Articulated Objects](https://mlsanity.com/paper/2412.07237/) (2024)
- [Fine-Grained Side Information Guided Dual-Prompts for Zero-Shot Skeleton Action Recognition](https://mlsanity.com/paper/2404.07487/) (2024)
- [SA-DVAE: Improving Zero-Shot Skeleton-Based Action Recognition by Disentangled Variational Autoencoders](https://mlsanity.com/paper/2407.13460/) (2024)
- [Learning Multimodal Data Augmentation in Feature Space](https://mlsanity.com/paper/2212.14453/) (2022)
- [Neuron: Learning Context-Aware Evolving Representations for Zero-Shot Skeleton Action Recognition](https://mlsanity.com/paper/2411.11288/) (2024)
- [Multi-Semantic Fusion Model for Generalized Zero-Shot Skeleton-Based Action Recognition](https://mlsanity.com/paper/2309.09592/) (2023)
- [Zero-Shot Skeleton-based Action Recognition with Dual Visual-Text Alignment](https://mlsanity.com/paper/2409.14336/) (2024)
- [Bridging the Skeleton-Text Modality Gap: Diffusion-Powered Modality Alignment for Zero-shot Skeleton-based Action Recognition](https://mlsanity.com/paper/2411.10745/) (2024)
- [A Survey on Visual Transfer Learning using Knowledge Graphs](https://mlsanity.com/paper/2201.11794/) (2022)
- [An Information Compensation Framework for Zero-Shot Skeleton-based Action Recognition](https://mlsanity.com/paper/2406.00639/) (2024)
- [CIZSL++: Creativity Inspired Generative Zero-Shot Learning](https://mlsanity.com/paper/2101.00173/) (2021)
- [A Gating Model for Bias Calibration in Generalized Zero-shot Learning](https://mlsanity.com/paper/2203.04195/) (2022)
- [EGANS: Evolutionary Generative Adversarial Network Search for Zero-Shot Learning](https://mlsanity.com/paper/2308.09915/) (2023)
- [Frequency-Semantic Enhanced Variational Autoencoder for Zero-Shot Skeleton-based Action Recognition](https://mlsanity.com/paper/2506.22179/) (2025)
- [Context-Conditional Adaptation for Recognizing Unseen Classes in Unseen Domains](https://mlsanity.com/paper/2107.07497/) (2021)
- [Generalized Zero-Shot Learning using Multimodal Variational Auto-Encoder with Semantic Concepts](https://mlsanity.com/paper/2106.14082/) (2021)
- [Structured Latent Embeddings for Recognizing Unseen Classes in Unseen Domains](https://mlsanity.com/paper/2107.05622/) (2021)
- [ZS-SLR: Zero-Shot Sign Language Recognition from RGB-D Videos](https://mlsanity.com/paper/2108.10059/) (2021)
- [Learning Aligned Cross-Modal Representation for Generalized Zero-Shot Classification](https://mlsanity.com/paper/2112.12927/) (2021)
- [Zero-Shot Skeleton-Based Action Recognition With Prototype-Guided Feature Alignment](https://mlsanity.com/paper/2507.00566/) (2025)
- [Better Generalized Few-Shot Learning Even Without Base Data](https://mlsanity.com/paper/2211.16095/) (2022)

## Abstract (arXiv)

Many of the existing methods for learning joint embedding of images and text use only supervised information from paired images and its textual attributes. Taking advantage of the recent success of unsupervised learning in deep neural networks, we propose an end-to-end learning framework that is able to extract more robust multi-modal representations across domains. The proposed method combines representation learning models (i.e., auto-encoders) together with cross-domain learning criteria (i.e., Maximum Mean Discrepancy loss) to learn joint embeddings for semantic and visual features. A novel technique of unsupervised-data adaptation inference is introduced to construct more comprehensive embeddings for both labeled and unlabeled data. We evaluate our method on Animals with Attributes and Caltech-UCSD Birds 200-2011 dataset with a wide range of applications, including zero and few-shot image recognition and retrieval, from inductive to transductive settings. Empirically, we show that our framework improves over the current state of the art on many of the considered tasks.