mlsanityData to 7 Oct 2026

Hierarchy-based Image Embeddings for Semantic Image Retrieval

arXiv 1809.09924 ↗First posted 26 Sep 2018cs.CV cs.AI cs.IRcited by 41

Not on the map. Why?

Abstract (arXiv)

Deep neural networks trained for classification have been found to learn powerful image representations, which are also often used for other tasks such as comparing images w.r.t. their visual similarity. However, visual similarity does not imply semantic similarity. In order to learn semantically discriminative features, we propose to map images onto class embeddings whose pair-wise dot products correspond to a measure of semantic similarity between classes. Such an embedding does not only improve image retrieval results, but could also facilitate integrating semantics for other tasks, e.g., novelty detection or few-shot learning. We introduce a deterministic algorithm for computing the class centroids directly based on prior world-knowledge encoded in a hierarchy of classes such as WordNet. Experiments on CIFAR-100, NABirds, and ImageNet show that our learned semantic image embeddings improve the semantic consistency of image retrieval results by a large margin.

Key facts. Cited by 41 papers in the map (data through 7 Oct 2026). First posted on arXiv 26 Sep 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 11; 2022: 7; 2023: 9; 2024: 5; 2025: 5; 2026: 4.

41 papers.

Sort:

By area:

  1. 2022Deep Hierarchical Semantic Segmentationcited by 84
  2. 2021Semantics for Robotic Mapping, Perception and Interaction: A Surveycited by 56
  3. 2021On Semantic Similarity in Video Retrievalcited by 45
  4. 2022Learning Hierarchy Aware Features for Reducing Mistake Severitycited by 26
  5. 2021No Cost Likelihood Manipulation at Test Time for Making Better Mistakes in Deep Networkscited by 21
  6. 2022Integrating Language Guidance into Vision-based Deep Metric Learningcited by 19
  7. 2023Inducing Neural Collapse to a Fixed Hierarchy-Aware Frame for Reducing Mistake Severitycited by 13
  8. 2024LCA-on-the-Line: Benchmarking Out-of-Distribution Generalization with Class Taxonomiescited by 12
  9. 2024SHiNe: Semantic Hierarchy Nexus for Open-vocabulary Object Detectioncited by 11
  10. 2024HELM: Hierarchical Encoding for mRNA Language Modelingcited by 9
  11. 2025Probabilistic Embeddings for Frozen Vision-Language Models: Uncertainty Quantification with Gaussian Process Latent Variable Modelscited by 9
  12. 2022Hierarchical Few-Shot Object Detection: Problem, Benchmark and Methodcited by 8
29 more
  1. 2025boldsymbolλ-Orthogonality Regularization for Compatible Representation Learningcited by 8
  2. 2021HIERMATCH: Leveraging Label Hierarchies for Improving Semi-Supervised Learningcited by 7
  3. 2022A Survey of Neural Treescited by 7
  4. 2023SEAL: Simultaneous Label Hierarchy Exploration And Learningcited by 6
  5. 2021Scene Graph Embeddings Using Relative Similarity Supervisioncited by 5
  6. 2021MMF: Multi-Task Multi-Structure Fusion for Hierarchical Image Classificationcited by 5
  7. 2023Test-Time Amendment with a Coarse Classifier for Fine-Grained Classificationcited by 5
  8. 2023Geometry-Aware Adaptation for Pretrained Modelscited by 5
  9. 2022Transformers in Action: Weakly Supervised Action Segmentationcited by 4
  10. 2021United We Learn Better: Harvesting Learning Improvements From Class Hierarchies Across Taskscited by 3
  11. 2021Encoding Hierarchical Information in Neural Networks helps in Subpopulation Shiftcited by 3
  12. 2023Integrating Visual and Semantic Similarity Using Hierarchies for Image Retrievalcited by 3
  13. 2024Learning Structured Representations by Embedding Class Hierarchy with Fast Optimal Transportcited by 3
  14. 2025SoilNet: A Multimodal Multitask Model for Hierarchical Classification of Soil Horizonscited by 2
  15. 2021A Hierarchical Assessment of Adversarial Severitycited by 1
  16. 2021Semantic-Based Few-Shot Learning by Interactive Psychometric Testingcited by 1
  17. 2022Is Medieval Distant Viewing Possible? : Extending and Enriching Annotation of Legacy Image Collections using Visual Analyticscited by 1
  18. 2023Inspecting class hierarchies in classification-based metric learning modelscited by 1
  19. 2023Beyond One-Hot-Encoding: Injecting Semantics to Drive Image Classifierscited by 1
  20. 2023DatasetEquity: Are All Samples Created Equal? In The Quest For Equity Within Datasetscited by 1
  21. 2024Enhanced document retrieval with topic embeddingscited by 1
  22. 2021Understanding top-down attention using task-oriented ablation designcited by 0
  23. 2023Are "Hierarchical" Visual Representations Hierarchical?cited by 0
  24. 2025Hier-COS: Making Deep Features Hierarchy-aware via Composition of Orthogonal Subspacescited by 0
  25. 2025Descriptive Image-Text Matching with Graded Contextual Similaritycited by 0
  26. 2026Physical Knot Classification Beyond Accuracy: A Benchmark and Diagnostic Studycited by 0
  27. 2026When Labels Have Structure: Improving Image Classification with Hierarchy-Aware Cross-Entropycited by 0
  28. 2026Online Continual Learning with Dynamic Label Hierarchiescited by 0
  29. 2026Subtype Robustness Is Not Just Accuracy: Calibration Under Unseen Subtype Shiftcited by 0