# Self-supervised Photographic Image Layout Representation Learning

arXiv: https://arxiv.org/abs/2403.03740 · HTML: https://mlsanity.com/paper/2403.03740/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 58 references that are in the map.
- First posted on arXiv 6 Mar 2024.

## Builds on (most cited first, 17 of 17)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [MUSIQ: Multi-scale Image Quality Transformer](https://mlsanity.com/paper/2108.05997/) (2021)
- [Segmentation Transformer: Object-Contextual Representations for Semantic Segmentation](https://mlsanity.com/paper/1909.11065/) (2019)
- [Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detection](https://mlsanity.com/paper/2006.04388/) (2020)
- [Graph Self-Supervised Learning: A Survey](https://mlsanity.com/paper/2103.00111/) (2021)
- [Photo Aesthetics Ranking Network with Attributes and Content Adaptation](https://mlsanity.com/paper/1606.01621/) (2016)
- [Personalized Image Aesthetics Assessment with Rich Attributes](https://mlsanity.com/paper/2203.16754/) (2022)
- [Single Image Depth Estimation: An Overview](https://mlsanity.com/paper/2104.06456/) (2021)
- [BigDetection: A Large-scale Benchmark for Improved Object Detector Pre-training](https://mlsanity.com/paper/2203.13249/) (2022)
- [Permutation-Invariant Variational Autoencoder for Graph-Level Representation Learning](https://mlsanity.com/paper/2104.09856/) (2021)
- [Image Composition Assessment with Saliency-augmented Multi-pattern Pooling](https://mlsanity.com/paper/2104.03133/) (2021)
- [LayoutGMN: Neural Graph Matching for Structural Layout Similarity](https://mlsanity.com/paper/2012.06547/) (2020)
- [Magic Layouts: Structural Prior for Component Detection in User Interface Designs](https://mlsanity.com/paper/2106.07615/) (2021)
- [Group Contrastive Self-Supervised Learning on Graphs](https://mlsanity.com/paper/2107.09787/) (2021)
- [ALR-GAN: Adaptive Layout Refinement for Text-to-Image Synthesis](https://mlsanity.com/paper/2304.06297/) (2023)
- [Intra-class Adaptive Augmentation with Neighbor Correction for Deep Metric Learning](https://mlsanity.com/paper/2211.16264/) (2022)
- [Aesthetic Photo Collage with Deep Reinforcement Learning](https://mlsanity.com/paper/2110.09775/) (2021)

## Built on by (most cited first, 3 of 3)

- [Semantic to Structure: Learning Structural Representations for Infringement Detection](https://mlsanity.com/paper/2502.07323/) (2025)
- [Semantically Stable Image Composition Analysis via Saliency and Gradient Vector Flow Fusion](https://mlsanity.com/paper/2604.16500/) (2026)
- [Learning visual representations for compositional analysis of artworks and photographs](https://mlsanity.com/paper/2608.06142/) (2026)

## Abstract (arXiv)

In the domain of image layout representation learning, the critical process of translating image layouts into succinct vector forms is increasingly significant across diverse applications, such as image retrieval, manipulation, and generation. Most approaches in this area heavily rely on costly labeled datasets and notably lack in adapting their modeling and learning methods to the specific nuances of photographic image layouts. This shortfall makes the learning process for photographic image layouts suboptimal. In our research, we directly address these challenges. We innovate by defining basic layout primitives that encapsulate various levels of layout information and by mapping these, along with their interconnections, onto a heterogeneous graph structure. This graph is meticulously engineered to capture the intricate layout information within the pixel domain explicitly. Advancing further, we introduce novel pretext tasks coupled with customized loss functions, strategically designed for effective self-supervised learning of these layout graphs. Building on this foundation, we develop an autoencoder-based network architecture skilled in compressing these heterogeneous layout graphs into precise, dimensionally-reduced layout representations. Additionally, we introduce the LODB dataset, which features a broader range of layout categories and richer semantics, serving as a comprehensive benchmark for evaluating the effectiveness of layout representation learning methods. Our extensive experimentation on this dataset demonstrates the superior performance of our approach in the realm of photographic image layout representation learning.