# Sketching Image Gist: Human-Mimetic Hierarchical Scene Graph Generation

arXiv: https://arxiv.org/abs/2007.08760 · HTML: https://mlsanity.com/paper/2007.08760/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Jul 2020.

## Built on by (most cited first, 19 of 19)

- [Bipartite Graph Network with Adaptive Message Passing for Unbiased Scene Graph Generation](https://mlsanity.com/paper/2104.00308/) (2021)
- [A Comprehensive Survey of Scene Graphs: Generation and Application](https://mlsanity.com/paper/2104.01111/) (2021)
- [Scene Graph Generation: A Comprehensive Survey](https://mlsanity.com/paper/2201.00443/) (2022)
- [Recovering the Unbiased Scene Graphs from the Biased Ones](https://mlsanity.com/paper/2107.02112/) (2021)
- [Learning to Generate Scene Graph from Natural Language Supervision](https://mlsanity.com/paper/2109.02227/) (2021)
- [Target Adaptive Context Aggregation for Video Scene Graph Generation](https://mlsanity.com/paper/2108.08121/) (2021)
- [Structured Sparse R-CNN for Direct Scene Graph Generation](https://mlsanity.com/paper/2106.10815/) (2021)
- [SwapMix: Diagnosing and Regularizing the Over-Reliance on Visual Context in Visual Question Answering](https://mlsanity.com/paper/2204.02285/) (2022)
- [RU-Net: Regularized Unrolling Network for Scene Graph Generation](https://mlsanity.com/paper/2205.01297/) (2022)
- [HiKER-SGG: Hierarchical Knowledge Enhanced Robust Scene Graph Generation](https://mlsanity.com/paper/2403.12033/) (2024)
- [Improving Scene Graph Classification by Exploiting Knowledge from Texts](https://mlsanity.com/paper/2102.04760/) (2021)
- [Leveraging Predicate and Triplet Learning for Scene Graph Generation](https://mlsanity.com/paper/2406.02038/) (2024)
- [Tackling the Challenges in Scene Graph Generation with Local-to-Global Interactions](https://mlsanity.com/paper/2106.08543/) (2021)
- [Explore Contextual Information for 3D Scene Graph Generation](https://mlsanity.com/paper/2210.06240/) (2022)
- [Fine-Grained is Too Coarse: A Novel Data-Centric Approach for Efficient Scene Graph Generation](https://mlsanity.com/paper/2305.18668/) (2023)
- [Topic Scene Graph Generation by Attention Distillation from Caption](https://mlsanity.com/paper/2110.05731/) (2021)
- [Context-aware Mixture-of-Experts for Unbiased Scene Graph Generation](https://mlsanity.com/paper/2208.07109/) (2022)
- [Zero-Shot Scene Graph Generation via Triplet Calibration and Reduction](https://mlsanity.com/paper/2309.03542/) (2023)
- [UniQ: Unified Decoder with Task-specific Queries for Efficient Scene Graph Generation](https://mlsanity.com/paper/2501.05687/) (2025)

## Abstract (arXiv)

Scene graph aims to faithfully reveal humans' perception of image content. When humans analyze a scene, they usually prefer to describe image gist first, namely major objects and key relations in a scene graph. This humans' inherent perceptive habit implies that there exists a hierarchical structure about humans' preference during the scene parsing procedure. Therefore, we argue that a desirable scene graph should be also hierarchically constructed, and introduce a new scheme for modeling scene graph. Concretely, a scene is represented by a human-mimetic Hierarchical Entity Tree (HET) consisting of a series of image regions. To generate a scene graph based on HET, we parse HET with a Hybrid Long Short-Term Memory (Hybrid-LSTM) which specifically encodes hierarchy and siblings context to capture the structured information embedded in HET. To further prioritize key relations in the scene graph, we devise a Relation Ranking Module (RRM) to dynamically adjust their rankings by learning to capture humans' subjective perceptive habits from objective entity saliency and size. Experiments indicate that our method not only achieves state-of-the-art performances for scene graph generation, but also is expert in mining image-specific relations which play a great role in serving downstream tasks.