# Target-Oriented Deformation of Visual-Semantic Embedding Space

arXiv: https://arxiv.org/abs/1910.06514 · HTML: https://mlsanity.com/paper/1910.06514/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Oct 2019.

## Built on by (most cited first, 2 of 2)

- [Contrastive Learning of Visual-Semantic Embeddings](https://mlsanity.com/paper/2110.08872/) (2021)
- [Active Mining Sample Pair Semantics for Image-text Matching](https://mlsanity.com/paper/2311.05425/) (2023)

## Abstract (arXiv)

Multimodal embedding is a crucial research topic for cross-modal understanding, data mining, and translation. Many studies have attempted to extract representations from given entities and align them in a shared embedding space. However, because entities in different modalities exhibit different abstraction levels and modality-specific information, it is insufficient to embed related entities close to each other. In this study, we propose the Target-Oriented Deformation Network (TOD-Net), a novel module that continuously deforms the embedding space into a new space under a given condition, thereby adjusting similarities between entities. Unlike methods based on cross-modal attention, TOD-Net is a post-process applied to the embedding space learned by existing embedding systems and improves their performances of retrieval. In particular, when combined with cutting-edge models, TOD-Net gains the state-of-the-art cross-modal retrieval model associated with the MSCOCO dataset. Qualitative analysis reveals that TOD-Net successfully emphasizes entity-specific concepts and retrieves diverse targets via handling higher levels of diversity than existing models.