Image-to-Image Retrieval by Learning Similarity between Scene Graphs
Not on the map. Why?
Abstract (arXiv)
As a scene graph compactly summarizes the high-level content of an image in a structured and symbolic manner, the similarity between scene graphs of two images reflects the relevance of their contents. Based on this idea, we propose a novel approach for image-to-image retrieval using scene graph similarity measured by graph neural networks. In our approach, graph neural networks are trained to predict the proxy image relevance measure, computed from human-annotated captions using a pre-trained sentence similarity model. We collect and publish the dataset for image relevance measured by human annotators to evaluate retrieval algorithms. The collected dataset shows that our method agrees well with the human perception of image similarity than other competitive baselines.
Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). First posted on arXiv 29 Dec 2020.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2022: 3; 2023: 6; 2024: 4; 2025: 6; 2026: 4.
23 papers.
Sort:
By area:
- 2023Relational Context Learning for Human-Object Interaction Detectioncited by 35
- 2023Dysen-VDM: Empowering Dynamics-aware Text-to-Video Diffusion with LLMscited by 35
- 2024HiKER-SGG: Hierarchical Knowledge Enhanced Robust Scene Graph Generationcited by 14
- 2025Locality-Aware Zero-Shot Human-Object Interaction Detectioncited by 12
- 2023Graph Neural Networks in Vision-Language Image Understanding: A Surveycited by 10
- 2025Statistical Confidence Rescoring for Robust 3D Scene Graph Generation from Multi-View Imagescited by 5
- 2024Automated Floodwater Depth Estimation Using Large Multimodal Model for Rapid Flood Mappingcited by 4
- 2023SPAN: Learning Similarity between Scene Graphs and Images with Transformerscited by 3
- 2023Generalized Unbiased Scene Graph Generationcited by 3
- 2024Frequency-based Matcher for Long-tailed Semantic Segmentationcited by 3
- 2025SATURN: Autoregressive Image Generation Guided by Scene Graphscited by 3
- 2022SPA-VAE: Similar-Parts-Assignment for Unsupervised 3D Point Cloud Generationcited by 2
11 more
- 2025SCENIR: Visual Semantic Clarity through Unsupervised Scene Graph Retrievalcited by 2
- 2026Response-G1: Explicit Scene Graph Modeling for Proactive Streaming Video Understandingcited by 2
- 2022Interactive Visual Pattern Search on Graph Data via Graph Representation Learningcited by 1
- 2022Training Free Graph Neural Networks for Graph Matchingcited by 0
- 2023VITR: Augmenting Vision Transformers with Relation-Focused Learning for Cross-Modal Information Retrievalcited by 0
- 2024Image Similarity using An Ensemble of Context-Sensitive Modelscited by 0
- 2025Multi-Modal Scene Graph with Kolmogorov-Arnold Experts for Audio-Visual Question Answeringcited by 0
- 2025Through the PRISm: Importance-Aware Scene Graphs for Image Retrievalcited by 0
- 2026VENUS: Visual Editing with Noise Inversion Using Scene Graphscited by 0
- 2026U-CECE: A Universal Multi-Resolution Framework for Conceptual Counterfactual Explanationscited by 0
- 2026TBSG-Net: Temporal Bipartite Scene Graph Network for Fine-Grained Video Moment Retrievalcited by 0
- 2026TBSG-Net: Temporal Bipartite Scene Graph Network for Fine-Grained Video Moment Retrieval3 Aug 2026
- 2026Response-G1: Explicit Scene Graph Modeling for Proactive Streaming Video Understanding8 May 2026
- 2026U-CECE: A Universal Multi-Resolution Framework for Conceptual Counterfactual Explanations9 Apr 2026
- 2026VENUS: Visual Editing with Noise Inversion Using Scene Graphs12 Jan 2026
- 2025Through the PRISm: Importance-Aware Scene Graphs for Image Retrieval20 Dec 2025
- 2025Multi-Modal Scene Graph with Kolmogorov-Arnold Experts for Audio-Visual Question Answering28 Nov 2025
- 2025SATURN: Autoregressive Image Generation Guided by Scene Graphs20 Aug 2025
- 2025Statistical Confidence Rescoring for Robust 3D Scene Graph Generation from Multi-View Images5 Aug 2025
- 2025Locality-Aware Zero-Shot Human-Object Interaction Detection26 May 2025
- 2025SCENIR: Visual Semantic Clarity through Unsupervised Scene Graph Retrieval21 May 2025
- 2024Frequency-based Matcher for Long-tailed Semantic Segmentation6 Jun 2024
- 2024HiKER-SGG: Hierarchical Knowledge Enhanced Robust Scene Graph Generation18 Mar 2024
11 more
- 2024Automated Floodwater Depth Estimation Using Large Multimodal Model for Rapid Flood Mapping26 Feb 2024
- 2024Image Similarity using An Ensemble of Context-Sensitive Models15 Jan 2024
- 2023Dysen-VDM: Empowering Dynamics-aware Text-to-Video Diffusion with LLMs26 Aug 2023
- 2023Generalized Unbiased Scene Graph Generation9 Aug 2023
- 2023Relational Context Learning for Human-Object Interaction Detection11 Apr 2023
- 2023SPAN: Learning Similarity between Scene Graphs and Images with Transformers2 Apr 2023
- 2023Graph Neural Networks in Vision-Language Image Understanding: A Survey7 Mar 2023
- 2023VITR: Augmenting Vision Transformers with Relation-Focused Learning for Cross-Modal Information Retrieval13 Feb 2023
- 2022SPA-VAE: Similar-Parts-Assignment for Unsupervised 3D Point Cloud Generation15 Mar 2022
- 2022Interactive Visual Pattern Search on Graph Data via Graph Representation Learning18 Feb 2022
- 2022Training Free Graph Neural Networks for Graph Matching14 Jan 2022
- 2025Locality-Aware Zero-Shot Human-Object Interaction Detectioncited by 12
- 2025Locality-Aware Zero-Shot Human-Object Interaction Detection26 May 2025
- 2023Relational Context Learning for Human-Object Interaction Detectioncited by 35
- 2024HiKER-SGG: Hierarchical Knowledge Enhanced Robust Scene Graph Generationcited by 14
- 2023Graph Neural Networks in Vision-Language Image Understanding: A Surveycited by 10
- 2025Statistical Confidence Rescoring for Robust 3D Scene Graph Generation from Multi-View Imagescited by 5
- 2024Automated Floodwater Depth Estimation Using Large Multimodal Model for Rapid Flood Mappingcited by 4
- 2023SPAN: Learning Similarity between Scene Graphs and Images with Transformerscited by 3
- 2023Generalized Unbiased Scene Graph Generationcited by 3
- 2024Frequency-based Matcher for Long-tailed Semantic Segmentationcited by 3
- 2025SATURN: Autoregressive Image Generation Guided by Scene Graphscited by 3
- 2022SPA-VAE: Similar-Parts-Assignment for Unsupervised 3D Point Cloud Generationcited by 2
- 2025SCENIR: Visual Semantic Clarity through Unsupervised Scene Graph Retrievalcited by 2
- 2026Response-G1: Explicit Scene Graph Modeling for Proactive Streaming Video Understandingcited by 2
12 most cited of 20 All 20 (JSON)
- 2026TBSG-Net: Temporal Bipartite Scene Graph Network for Fine-Grained Video Moment Retrieval3 Aug 2026
- 2026Response-G1: Explicit Scene Graph Modeling for Proactive Streaming Video Understanding8 May 2026
- 2026U-CECE: A Universal Multi-Resolution Framework for Conceptual Counterfactual Explanations9 Apr 2026
- 2026VENUS: Visual Editing with Noise Inversion Using Scene Graphs12 Jan 2026
- 2025Through the PRISm: Importance-Aware Scene Graphs for Image Retrieval20 Dec 2025
- 2025SATURN: Autoregressive Image Generation Guided by Scene Graphs20 Aug 2025
- 2025Statistical Confidence Rescoring for Robust 3D Scene Graph Generation from Multi-View Images5 Aug 2025
- 2025SCENIR: Visual Semantic Clarity through Unsupervised Scene Graph Retrieval21 May 2025
- 2024Frequency-based Matcher for Long-tailed Semantic Segmentation6 Jun 2024
- 2024HiKER-SGG: Hierarchical Knowledge Enhanced Robust Scene Graph Generation18 Mar 2024
- 2024Automated Floodwater Depth Estimation Using Large Multimodal Model for Rapid Flood Mapping26 Feb 2024
- 2024Image Similarity using An Ensemble of Context-Sensitive Models15 Jan 2024
12 newest of 20 All 20 (JSON)