mlsanityData to 7 Oct 2026

Structured Query-Based Image Retrieval Using Scene Graphs

arXiv 2005.06653 ↗First posted 13 May 2020cs.CV cs.LGcited by 33

Not on the map. Why?

Abstract (arXiv)

A structured query can capture the complexity of object interactions (e.g. 'woman rides motorcycle') unlike single objects (e.g. 'woman' or 'motorcycle'). Retrieval using structured queries therefore is much more useful than single object retrieval, but a much more challenging problem. In this paper we present a method which uses scene graph embeddings as the basis for an approach to image retrieval. We examine how visual relationships, derived from scene graphs, can be used as structured queries. The visual relationships are directed subgraphs of the scene graph with a subject and object as nodes connected by a predicate relationship. Notably, we are able to achieve high recall even on low to medium frequency objects found in the long-tailed COCO-Stuff dataset, and find that adding a visual relationship-inspired loss boosts our recall by 10% in the best case.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 May 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 10; 2023: 6; 2024: 9; 2025: 3; 2026: 2.

33 papers.

Sort:

By area:

  1. 2021A Comprehensive Survey of Scene Graphs: Generation and Applicationcited by 125
  2. 2021Hierarchical Representations and Explicit Memory: Learning Effective Navigation Policies on 3D Scene Graphs using Graph Neural Networkscited by 50
  3. 2023Prototype-based Embedding Network for Scene Graph Generationcited by 37
  4. 2022Fine-Grained Predicates Learning for Scene Graph Generationcited by 27
  5. 2022Integrating Object-aware and Interaction-aware Knowledge for Weakly Supervised Scene Graph Generationcited by 20
  6. 2021BGT-Net: Bidirectional GRU Transformer Network for Scene Graph Generationcited by 18
  7. 2022Unbiased Heterogeneous Scene Graph Generation with Relation-aware Message Passing Neural Networkcited by 17
  8. 2022Adaptive Fine-Grained Predicates Learning for Scene Graph Generationcited by 14
  9. 2024LLaVA-SpaceSGG: Visual Instruct Tuning for Open-vocabulary Scene Graph Generation with Enhanced Spatial Relationscited by 13
  10. 2022Dual-branch Hybrid Learning Network for Unbiased Scene Graph Generationcited by 10
  11. 2022Rethinking the Evaluation of Unbiased Scene Graph Generationcited by 9
  12. 2023Explainable Action Prediction through Self-Supervision on Scene Graphscited by 9
21 more
  1. 2023Weakly-Supervised Learning of Visual Relations in Multimodal Pretrainingcited by 6
  2. 2024ROOT: VLM based System for Indoor Scene Understanding and Beyondcited by 6
  3. 2024RA-SGG: Retrieval-Augmented Scene Graph Generation Framework via Multi-Prototype Learningcited by 6
  4. 2024AUG: A New Dataset and An Efficient Model for Aerial Image Urban Scene Graph Generationcited by 5
  5. 2025Statistical Confidence Rescoring for Robust 3D Scene Graph Generation from Multi-View Imagescited by 5
  6. 2022Visual Semantic Parsing: From Images to Abstract Meaning Representationcited by 4
  7. 2024Scene Graph Generation Strategy with Co-occurrence Knowledge and Learnable Term Frequencycited by 4
  8. 2025Weakly Supervised Video Scene Graph Generation via Natural Language Supervisioncited by 4
  9. 2023SPAN: Learning Similarity between Scene Graphs and Images with Transformerscited by 3
  10. 2023Generalized Unbiased Scene Graph Generationcited by 3
  11. 2024ReCon1M:A Large-scale Benchmark Dataset for Relation Comprehension in Remote Sensing Imagerycited by 3
  12. 2022Scene Graph Modification as Incremental Structure Expandingcited by 2
  13. 2022SGRAM: Improving Scene Graph Parsing via Abstract Meaning Representationcited by 2
  14. 2026Response-G1: Explicit Scene Graph Modeling for Proactive Streaming Video Understandingcited by 2
  15. 2022Interactive Visual Pattern Search on Graph Data via Graph Representation Learningcited by 1
  16. 2024Fine-Grained Scene Graph Generation via Sample-Level Bias Predictioncited by 1
  17. 2025MSTAR: Box-free Multi-query Scene Text Retrieval with Attention Recyclingcited by 1
  18. 2023Head-Tail Cooperative Learning Network for Unbiased Scene Graph Generationcited by 0
  19. 2024On Support Relations Inference and Scene Hierarchy Graph Construction from Point Cloud in Clustered Environmentscited by 0
  20. 2024A transition towards virtual representations of visual scenescited by 0
  21. 2026SGG-Rrm 3: From Next-Token Prediction to End-to-End Unbiased Scene Graph Generationcited by 0