mlsanityData to 14 Oct 2026

Information-Theoretic Hashing for Zero-Shot Cross-Modal Retrieval

arXiv 2209.12491 ↗First posted 26 Sep 2022cs.LG cs.IRcited by 0

Not on the map. Why?

Abstract (arXiv)

Zero-shot cross-modal retrieval (ZS-CMR) deals with the retrieval problem among heterogenous data from unseen classes. Typically, to guarantee generalization, the pre-defined class embeddings from natural language processing (NLP) models are used to build a common space. In this paper, instead of using an extra NLP model to define a common space beforehand, we consider a totally different way to construct (or learn) a common hamming space from an information-theoretic perspective. We term our model the Information-Theoretic Hashing (ITH), which is composed of two cascading modules: an Adaptive Information Aggregation (AIA) module; and a Semantic Preserving Encoding (SPE) module. Specifically, our AIA module takes the inspiration from the Principle of Relevant Information (PRI) to construct a common space that adaptively aggregates the intrinsic semantics of different modalities of data and filters out redundant or irrelevant information. On the other hand, our SPE module further generates the hashing codes of different modalities by preserving the similarity of intrinsic semantics with the element-wise Kullback-Leibler (KL) divergence. A total correlation regularization term is also imposed to reduce the redundancy amongst different dimensions of hash codes. Sufficient experiments on three benchmark datasets demonstrate the superiority of the proposed ITH in ZS-CMR. Source code is available in the supplementary material.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 25 of its 56 references that are in the map. First posted on arXiv 26 Sep 2022.

Builds on

25 of 56 references.

References · 25

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  4. 2015Deep Learning and the Information Bottleneck Principlecited by 802
  5. 2014Distributed Representations of Sentences and Documentscited by 678
  6. 2021Masked Feature Prediction for Self-Supervised Visual Pre-Trainingcited by 576
  7. 2020TERA: Self-Supervised Learning of Transformer Encoder Representation for Speechcited by 188
  8. 2016Learning a Deep Embedding Model for Zero-Shot Learningcited by 153
  9. 2020Cross-domain Correspondence Learning for Exemplar-based Image Translationcited by 116
  10. 2019Semantics-aware BERT for Language Understandingcited by 112
  11. 2017Semantic Image Synthesis via Adversarial Learningcited by 87
  12. 2021Cross Modal Retrieval with Querybank Normalisationcited by 72
13 more
  1. 2016A Comprehensive Survey on Cross-modal Retrievalcited by 62
  2. 2021HSVA: Hierarchical Semantic-Visual Adaptation for Zero-Shot Learningcited by 57
  3. 2016Deep Cross-Modal Hashingcited by 54
  4. 2018Self-Supervised Adversarial Hashing Networks for Cross-Modal Retrievalcited by 29
  5. 2020Creating Something from Nothing: Unsupervised Knowledge Distillation for Cross-Modal Hashingcited by 26
  6. 2017An Overview of Cross-media Retrieval: Concepts, Methodologies, Benchmarks and Challengescited by 21
  7. 2020Comprehensive Graph-conditional Similarity Preserving Network for Unsupervised Cross-modal Hashingcited by 15
  8. 2017MHTN: Modal-adversarial Hybrid Transfer Network for Cross-modal Retrievalcited by 12
  9. 2018Cycle-Consistent Deep Generative Hashing for Cross-Modal Retrievalcited by 11
  10. 2018Attribute-Guided Network for Cross-Modal Zero-Shot Hashingcited by 10
  11. 2018Multivariate Extension of Matrix-based Renyi's α-order Entropy Functionalcited by 6
  12. 2019Cross-modal Zero-shot Hashingcited by 4
  13. 2020PRI-VAE: Principle-of-Relevant-Information Variational Autoencoderscited by 4

Built on by

Not cited yet.