mlsanityData to 7 Oct 2026

The More You Know: Using Knowledge Graphs for Image Classification

arXiv 1612.04844 ↗First posted 14 Dec 2016cs.CVcited by 63

Not on the map. Why?

Abstract (arXiv)

One characteristic that sets humans apart from modern learning-based computer vision algorithms is the ability to acquire knowledge about the world and use that knowledge to reason about the visual world. Humans can learn about the characteristics of objects and the relationships that occur between them to learn a large variety of visual concepts, often with few examples. This paper investigates the use of structured prior knowledge in the form of knowledge graphs and shows that using this knowledge improves performance on image classification. We build on recent work on end-to-end learning on graphs, introducing the Graph Search Neural Network as a way of efficiently incorporating large knowledge graphs into a vision classification pipeline. We show in a number of experiments that our method outperforms standard neural network baselines for multi-label classification.

Key facts. Cited by 63 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 Dec 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 20; 2022: 20; 2023: 11; 2024: 4; 2025: 7; 2026: 1.

63 papers.

Sort:

By area:

  1. 2022Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questionscited by 116
  2. 2022Align, Reason and Learn: Enhancing Medical Vision-and-Language Pre-training with Knowledgecited by 51
  3. 2021Hierarchical Object-to-Zone Graph for Object Navigationcited by 34
  4. 2022TIDEE: Tidying Up Novel Rooms using Visuo-Semantic Commonsense Priorscited by 30
  5. 2021Dual Graph Convolutional Networks with Transformer and Curriculum Learning for Image Captioningcited by 27
  6. 2021Act Like a Radiologist: Towards Reliable Multi-view Correspondence Reasoning for Mammogram Mass Detectioncited by 22
  7. 2023Multimodal Data Integration for Oncology in the Era of Deep Neural Networks: A Reviewcited by 20
  8. 2021Network Representation Learning: From Preprocessing, Feature Extraction to Node Embeddingcited by 19
  9. 2021The Pursuit of Knowledge: Discovering and Localizing Novel Categories using Dual Memorycited by 13
  10. 2021GraphFM: Graph Factorization Machines for Feature Interaction Modelingcited by 13
  11. 2025Taxonomy-Aware Evaluation of Vision-Language Modelscited by 11
  12. 2021Faster-LTN: a neuro-symbolic, end-to-end object detection architecturecited by 10
38 more
  1. 2021Applications of knowledge graphs for food science and industrycited by 8
  2. 2023Sample-Efficient Learning of Novel Visual Conceptscited by 8
  3. 2021Survey of Image Based Graph Neural Networkscited by 7
  4. 2022A Survey on Visual Transfer Learning using Knowledge Graphscited by 7
  5. 2022Duality-Induced Regularizer for Semantic Matching Knowledge Graph Embeddingscited by 7
  6. 2021DyGCN: Dynamic Graph Embedding with Graph Convolutional Networkcited by 6
  7. 2023Increasing Coverage and Precision of Textual Information in Multilingual Knowledge Graphscited by 6
  8. 2021GMLP: Building Scalable and Flexible Graph Neural Networks with Feature-Message Passingcited by 5
  9. 2021Magic Layouts: Structural Prior for Component Detection in User Interface Designscited by 5
  10. 2022KTN: Knowledge Transfer Network for Learning Multi-person 2D-3D Correspondencescited by 5
  11. 2024Capturing Knowledge Graphs and Rules with Octagon Embeddingscited by 5
  12. 2021Addressing materials' microstructure diversity using transfer learningcited by 4
  13. 2021Ontology-based n-ball Concept Embeddings Informing Few-shot Image Classificationcited by 4
  14. 2022DRAG: Dynamic Region-Aware GCN for Privacy-Leaking Image Detectioncited by 4
  15. 2022Brainish: Formalizing A Multimodal Language for Intelligence and Consciousnesscited by 4
  16. 2022ER: Equivariance Regularizer for Knowledge Graph Completioncited by 4
  17. 2022Content-based Graph Privacy Advisorcited by 4
  18. 2022Graph Neural Networks in Computer Vision -- Architectures, Datasets and Common Approachescited by 4
  19. 2023BB-GCN: A Bi-modal Bridged Graph Convolutional Network for Multi-label Chest X-Ray Recognitioncited by 4
  20. 2023InferTurbo: A Scalable System for Boosting Full-graph Inference of Graph Neural Network over Huge Graphscited by 4
  21. 2023Knowledge-Guided Short-Context Action Anticipation in Human-Centric Videoscited by 4
  22. 2023Reviving the Context: Camera Trap Species Classification as Link Prediction on Multimodal Knowledge Graphscited by 4
  23. 2021A Survey of Knowledge Enhanced Pre-trained Modelscited by 3
  24. 2022Learning Commonsense-aware Moment-Text Alignment for Fast Video Temporal Groundingcited by 3
  25. 2022A survey on the development status and application prospects of knowledge graph in smart gridscited by 2
  26. 2022Semantic Enhanced Knowledge Graph for Large-Scale Zero-Shot Learningcited by 2
  27. 2023Spatio-Temporal driven Attention Graph Neural Network with Block Adjacency matrix (STAG-NN-BA) for Remote Land-use Change Detectioncited by 2
  28. 2023Enabling High-Level Machine Reasoning with Cognitive Neuro-Symbolic Systemscited by 2
  29. 2025Learning Privacy from Visual Entitiescited by 2
  30. 2025Graph Your Own Promptcited by 2
  31. 2025Object-Centric Representation Learning for Enhanced 3D Semantic Scene Graph Predictioncited by 2
  32. 2021KGE-CL: Contrastive Learning of Tensor Decomposition Based Knowledge Graph Embeddingscited by 1
  33. 2023Teach Me How to Learn: A Perspective Review towards User-centered Neuro-symbolic Learning for Robotic Surgical Systemscited by 1
  34. 2024Stitching Gaps: Fusing Situated Perceptual Knowledge with Vision Transformers for High-Level Image Classificationcited by 1
  35. 2024Help Me Identify: Is an LLM+VQA System All We Need to Identify Visual Concepts?cited by 1
  36. 2025Segment Any Vehicle: Semantic and Visual Context Driven SAM and A Benchmarkcited by 1
  37. 2021Meta-Path-Free Representation Learning on Heterogeneous Networkscited by 0
  38. 2021Wider Vision: Enriching Convolutional Neural Networks via Alignment to External Knowledge Basescited by 0

50 most cited of 63 All 63 (JSON)