mlsanityData to 7 Oct 2026

Learning Context-Conditioned Predicate Semantics via Prototype Feedback

arXiv 2605.29610 ↗First posted 28 May 2026cs.CV cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

In scene graph generation, a central challenge is modeling polysemous predicates whose meanings shift across contexts. Prior approaches address this issue by decomposing predicates into multiple static prototypes or retrieving semantically similar exemplars. However, these strategies keep predicate representations static and cannot reorganize semantics to reflect image-specific evidence, leading to systematic confusions in ambiguous contexts. We propose AlignG, which learns context-conditioned predicate semantics via prototype feedback. AlignG infers context-conditioned predicate semantics from the relation candidates within each image and feeds the adapted semantics back to recalibrate relation representations. The learning objective anchors this adaptation to global semantic centers, preventing semantic drift while still allowing selective reorganization when the scene provides consistent relational cues. Experiments on VG-150 and GQA-200 show consistent improvements over state-of-the-art baselines, with F@100 improvements of +1.4 on VG-150 and +2.7 on GQA-200 under SGDet. We further visualize per-image prototype similarity shifts and observe coherent context-dependent reorganization where prototypes selectively merge or separate predicates according to scene evidence. The code is available at https://github.com/Namgyu97/AlignG-SGG.pytorch.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 24 of its 31 references that are in the map. First posted on arXiv 28 May 2026.

Builds on

24 of 31 references.

References · 24

  1. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  2. 2017Scene Graph Generation by Iterative Message Passingcited by 462
  3. 2017Neural Motifs: Scene Graph Parsing with Global Contextcited by 413
  4. 2020Unbiased Scene Graph Generation from Biased Trainingcited by 355
  5. 2018Learning to Compose Dynamic Tree Structures for Visual Contextscited by 258
  6. 2020GPS-Net: Graph Property Sensing Network for Scene Graph Generationcited by 165
  7. 2021Bipartite Graph Network with Adaptive Message Passing for Unbiased Scene Graph Generationcited by 137
  8. 2017Visual Translation Embedding Network for Visual Relation Detectioncited by 112
  9. 2021SGTR: End-to-end Scene Graph Generation with Transformercited by 102
  10. 2020Bridging Knowledge Graphs to Generate Scene Graphscited by 86
  11. 2020CogTree: Cognition Tree Loss for Unbiased Scene Graph Generationcited by 62
  12. 2022The Devil is in the Labels: Noisy Label Correction for Robust Scene Graph Generationcited by 60
12 more
  1. 2022Fine-Grained Scene Graph Generation with Data Transfercited by 57
  2. 2024EGTR: Extracting Graph from Transformer for Scene Graph Generationcited by 45
  3. 2023Prototype-based Embedding Network for Scene Graph Generationcited by 37
  4. 2023LLM4SGG: Large Language Models for Weakly Supervised Scene Graph Generationcited by 30
  5. 2023Devil's on the Edges: Selective Quad Attention for Scene Graph Generationcited by 23
  6. 2022Unbiased Heterogeneous Scene Graph Generation with Relation-aware Message Passing Neural Networkcited by 17
  7. 2023Compositional Feature Augmentation for Unbiased Scene Graph Generationcited by 15
  8. 2024HiKER-SGG: Hierarchical Knowledge Enhanced Robust Scene Graph Generationcited by 14
  9. 2024Adaptive Self-training Framework for Fine-grained Scene Graph Generationcited by 11
  10. 2024RA-SGG: Retrieval-Augmented Scene Graph Generation Framework via Multi-Prototype Learningcited by 6
  11. 2024Scene Graph Generation Strategy with Co-occurrence Knowledge and Learnable Term Frequencycited by 4
  12. 2024Semantic Diversity-aware Prototype-based Learning for Unbiased Scene Graph Generationcited by 4

Built on by

Not cited yet.