Pixels to Graphs by Associative Embedding
Not on the map. Why?
Abstract (arXiv)
Graphs are a useful abstraction of image content. Not only can graphs represent details about individual objects in a scene but they can capture the interactions between pairs of objects. We present a method for training a convolutional neural network such that it takes in an input image and produces a full graph definition. This is done end-to-end in a single stage with the use of associative embeddings. The network learns to simultaneously identify all of the elements that make up a graph and piece them together. We benchmark on the Visual Genome dataset, and demonstrate state-of-the-art performance on the challenging task of scene graph generation.
Key facts. Cited by 58 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Jun 2017.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 23; 2022: 10; 2023: 15; 2024: 4; 2025: 4; 2026: 2.
58 papers.
Sort:
By area:
- 2021SceneGraphFusion: Incremental 3D Scene Graph Prediction from RGB-D Sequencescited by 159
- 2021A Comprehensive Survey of Scene Graphs: Generation and Applicationcited by 125
- 2022RelTR: Relation Transformer for Scene Graph Generationcited by 95
- 2021Graph-to-3D: End-to-End Generation and Manipulation of 3D Scenes Using Scene Graphscited by 83
- 2022Scene Graph Generation: A Comprehensive Surveycited by 73
- 2022Relationformer: A Unified Framework for Image-to-Graph Generationcited by 67
- 2021Fully Convolutional Scene Graph Generationcited by 57
- 2021Learning latent causal graphs via mixture oraclescited by 55
- 2021From General to Specific: Informative Scene Graph Generation via Balance Adjustmentcited by 51
- 2021Exploiting Edge-Oriented Reasoning for 3D Point-based Scene Graph Analysiscited by 47
- 2024EGTR: Extracting Graph from Transformer for Scene Graph Generationcited by 45
- 2021Structured Sparse R-CNN for Direct Scene Graph Generationcited by 44
38 more
- 2023SceneGenie: Scene Graph Guided Diffusion Models for Image Synthesiscited by 41
- 2021Linguistic Structures as Weak Supervision for Visual Scene Graph Generationcited by 29
- 2021Segmentation-grounded Scene Graph Generationcited by 25
- 2022Iterative Scene Graph Generationcited by 24
- 2023Devil's on the Edges: Selective Quad Attention for Scene Graph Generationcited by 23
- 2021Unconditional Scene Graph Generationcited by 22
- 2021Exploiting Scene Graphs for Human-Object Interaction Detectioncited by 22
- 2021Zero-Shot Scene Graph Relation Prediction through Commonsense Knowledge Integrationcited by 21
- 2023Towards a Unified Transformer-based Framework for Scene Graph Generation and Human-object Interaction Detectioncited by 15
- 2021Semantic Compositional Learning for Low-shot Scene Graph Generationcited by 12
- 2024DSGG: Dense Relation Transformer for an End-to-end Scene Graph Generationcited by 11
- 2025Universal Scene Graph Generationcited by 11
- 2021Radar Camera Fusion via Representation Learning in Autonomous Drivingcited by 9
- 2021RelTransformer: A Transformer-Based Long-Tail Visual Relationship Recognitioncited by 9
- 2022Rethinking the Evaluation of Unbiased Scene Graph Generationcited by 9
- 2025Reusing Attention for One-stage Lane Topology Understandingcited by 9
- 2021GKNet: grasp keypoint network for grasp candidates detectioncited by 8
- 2022One-shot Scene Graph Generationcited by 8
- 2023Haystack: A Panoptic Scene Graph Dataset to Evaluate Rare Predicate Classescited by 8
- 2021End-to-End Hierarchical Relation Extraction for Generic Form Understandingcited by 7
- 2021MIGS: Meta Image Generation from Scene Graphscited by 7
- 2023Enhancing Scene Graph Generation with Hierarchical Relationships and Commonsense Knowledgecited by 7
- 2024Towards Lifelong Scene Graph Generation with Knowledge-ware In-context Prompt Learningcited by 6
- 2021Scenes and Surroundings: Scene Graph Generation using Relation Transformercited by 5
- 2021Hierarchical Lov'asz Embeddings for Proposal-free Panoptic Segmentationcited by 4
- 2022Explore Contextual Information for 3D Scene Graph Generationcited by 4
- 2022DisPositioNet: Disentangled Pose and Identity in Semantic Image Manipulationcited by 4
- 2022Relation Regularized Scene Graph Generationcited by 3
- 2023Hierarchical Relationships: A New Perspective to Enhance Scene Graph Generationcited by 3
- 2023Taking A Closer Look at Visual Relation: Unbiased Video Scene Graph Generation with Decoupled Label Learningcited by 3
- 2023LANDMARK: Language-guided Representation Enhancement Framework for Scene Graph Generationcited by 2
- 2023Unsupervised Traffic Scene Generation with Synthetic 3D Scene Graphscited by 2
- 2023Informative Scene Graph Generation via Debiasingcited by 2
- 2023RepSGG: Novel Representations of Entities and Relationships for Scene Graph Generationcited by 2
- 2024A Review and Efficient Implementation of Scene Graph Generation Metricscited by 2
- 2021Scene Graph Generation for Better Image Captioning?cited by 1
- 2022Biasing Like Human: A Cognitive Bias Framework for Scene Graph Generationcited by 1
- 2023Triple Correlations-Guided Label Supplementation for Unbiased Video Scene Graph Generationcited by 1
50 most cited of 58 All 58 (JSON)
- 2026Modular Diffusion Models for Structured Visual Recognition21 Jun 2026
- 2026Contrastive Multi-Modal Hypergraph Reasoning for 3D Crowd Mesh Recovery1 Apr 2026
- 2025Reusing Attention for One-stage Lane Topology Understanding23 Jul 2025
- 2025CoPa-SG: Dense Scene Graphs with Parametric and Proto-Relations26 Jun 2025
- 2025Universal Scene Graph Generation19 Mar 2025
- 2025Salient Temporal Encoding for Dynamic Scene Graph Generation15 Mar 2025
- 2024A Review and Efficient Implementation of Scene Graph Generation Metrics15 Apr 2024
- 2024EGTR: Extracting Graph from Transformer for Scene Graph Generation2 Apr 2024
- 2024DSGG: Dense Relation Transformer for an End-to-end Scene Graph Generation21 Mar 2024
- 2024Towards Lifelong Scene Graph Generation with Knowledge-ware In-context Prompt Learning26 Jan 2024
- 2023Enhancing Scene Graph Generation with Hierarchical Relationships and Commonsense Knowledge21 Nov 2023
- 2023Two Stream Scene Understanding on Graph Embedding12 Nov 2023
38 more
- 2023Towards a Unified Transformer-based Framework for Scene Graph Generation and Human-object Interaction Detection3 Nov 2023
- 2023A Novel Neural-symbolic System under Statistical Relational Learning16 Sep 2023
- 2023STDG: Semi-Teacher-Student Training Paradigram for Depth-guided One-stage Scene Graph Generation15 Sep 2023
- 2023RepSGG: Novel Representations of Entities and Relationships for Scene Graph Generation6 Sep 2023
- 2023Haystack: A Panoptic Scene Graph Dataset to Evaluate Rare Predicate Classes5 Sep 2023
- 2023Informative Scene Graph Generation via Debiasing10 Aug 2023
- 2023Triple Correlations-Guided Label Supplementation for Unbiased Video Scene Graph Generation30 Jul 2023
- 2023SceneGenie: Scene Graph Guided Diffusion Models for Image Synthesis28 Apr 2023
- 2023Devil's on the Edges: Selective Quad Attention for Scene Graph Generation7 Apr 2023
- 2023Taking A Closer Look at Visual Relation: Unbiased Video Scene Graph Generation with Decoupled Label Learning23 Mar 2023
- 2023Unsupervised Traffic Scene Generation with Synthetic 3D Scene Graphs15 Mar 2023
- 2023Hierarchical Relationships: A New Perspective to Enhance Scene Graph Generation13 Mar 2023
- 2023LANDMARK: Language-guided Representation Enhancement Framework for Scene Graph Generation2 Mar 2023
- 2022DisPositioNet: Disentangled Pose and Identity in Semantic Image Manipulation10 Nov 2022
- 2022Explore Contextual Information for 3D Scene Graph Generation12 Oct 2022
- 2022Rethinking the Evaluation of Unbiased Scene Graph Generation3 Aug 2022
- 2022Iterative Scene Graph Generation27 Jul 2022
- 2022Relationformer: A Unified Framework for Image-to-Graph Generation19 Mar 2022
- 2022Biasing Like Human: A Cognitive Bias Framework for Scene Graph Generation17 Mar 2022
- 2022Relation Regularized Scene Graph Generation22 Feb 2022
- 2022One-shot Scene Graph Generation22 Feb 2022
- 2022RelTR: Relation Transformer for Scene Graph Generation27 Jan 2022
- 2022Scene Graph Generation: A Comprehensive Survey3 Jan 2022
- 2021Scene Graph Generation with Geometric Context25 Nov 2021
- 2021MIGS: Meta Image Generation from Scene Graphs22 Oct 2021
- 2021Scene Graph Generation for Better Image Captioning?23 Sep 2021
- 2021From General to Specific: Informative Scene Graph Generation via Balance Adjustment30 Aug 2021
- 2021Graph-to-3D: End-to-End Generation and Manipulation of 3D Scenes Using Scene Graphs19 Aug 2021
- 2021Semantic Compositional Learning for Low-shot Scene Graph Generation19 Aug 2021
- 2021Exploiting Scene Graphs for Human-Object Interaction Detection19 Aug 2021
- 2021Unconditional Scene Graph Generation12 Aug 2021
- 2021Scenes and Surroundings: Scene Graph Generation using Relation Transformer12 Jul 2021
- 2021Zero-Shot Scene Graph Relation Prediction through Commonsense Knowledge Integration11 Jul 2021
- 2021Learning latent causal graphs via mixture oracles29 Jun 2021
- 2021Structured Sparse R-CNN for Direct Scene Graph Generation21 Jun 2021
- 2021GKNet: grasp keypoint network for grasp candidates detection16 Jun 2021
- 2021Hierarchical Lov'asz Embeddings for Proposal-free Panoptic Segmentation8 Jun 2021
- 2021End-to-End Hierarchical Relation Extraction for Generic Form Understanding2 Jun 2021
50 newest of 58 All 58 (JSON)
- 2021SceneGraphFusion: Incremental 3D Scene Graph Prediction from RGB-D Sequencescited by 159
- 2021A Comprehensive Survey of Scene Graphs: Generation and Applicationcited by 125
- 2022RelTR: Relation Transformer for Scene Graph Generationcited by 95
- 2021Graph-to-3D: End-to-End Generation and Manipulation of 3D Scenes Using Scene Graphscited by 83
- 2022Scene Graph Generation: A Comprehensive Surveycited by 73
- 2022Relationformer: A Unified Framework for Image-to-Graph Generationcited by 67
- 2021Fully Convolutional Scene Graph Generationcited by 57
- 2021Learning latent causal graphs via mixture oraclescited by 55
- 2021From General to Specific: Informative Scene Graph Generation via Balance Adjustmentcited by 51
- 2021Exploiting Edge-Oriented Reasoning for 3D Point-based Scene Graph Analysiscited by 47
- 2024EGTR: Extracting Graph from Transformer for Scene Graph Generationcited by 45
- 2021Structured Sparse R-CNN for Direct Scene Graph Generationcited by 44
12 most cited of 55 All 55 (JSON)
- 2026Contrastive Multi-Modal Hypergraph Reasoning for 3D Crowd Mesh Recovery1 Apr 2026
- 2025Reusing Attention for One-stage Lane Topology Understanding23 Jul 2025
- 2025CoPa-SG: Dense Scene Graphs with Parametric and Proto-Relations26 Jun 2025
- 2025Universal Scene Graph Generation19 Mar 2025
- 2025Salient Temporal Encoding for Dynamic Scene Graph Generation15 Mar 2025
- 2024A Review and Efficient Implementation of Scene Graph Generation Metrics15 Apr 2024
- 2024EGTR: Extracting Graph from Transformer for Scene Graph Generation2 Apr 2024
- 2024DSGG: Dense Relation Transformer for an End-to-end Scene Graph Generation21 Mar 2024
- 2024Towards Lifelong Scene Graph Generation with Knowledge-ware In-context Prompt Learning26 Jan 2024
- 2023Enhancing Scene Graph Generation with Hierarchical Relationships and Commonsense Knowledge21 Nov 2023
- 2023Two Stream Scene Understanding on Graph Embedding12 Nov 2023
- 2023Towards a Unified Transformer-based Framework for Scene Graph Generation and Human-object Interaction Detection3 Nov 2023
12 newest of 55 All 55 (JSON)