# Aesthetic Photo Collage with Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2110.09775 · HTML: https://mlsanity.com/paper/2110.09775/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 38 references that are in the map.
- First posted on arXiv 19 Oct 2021.

## Builds on (most cited first, 12 of 12)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Movie Description](https://mlsanity.com/paper/1605.03705/) (2016)
- [Distort-and-Recover: Color Enhancement using Deep Reinforcement Learning](https://mlsanity.com/paper/1804.04450/) (2018)
- [Crafting a Toolchain for Image Restoration by Deep Reinforcement Learning](https://mlsanity.com/paper/1804.03312/) (2018)
- [Reliable and Efficient Image Cropping: A Grid Anchor based Approach](https://mlsanity.com/paper/1904.04441/) (2019)
- [A-Lamp: Adaptive Layout-Aware Multi-Patch Deep Convolutional Neural Network for Photo Aesthetic Assessment](https://mlsanity.com/paper/1704.00248/) (2017)
- [Learning to Compose with Professional Photographs on the Web](https://mlsanity.com/paper/1702.00503/) (2017)
- [Deep Saliency with Encoded Low level Distance Map and High Level Features](https://mlsanity.com/paper/1604.05495/) (2016)
- [Deep Cropping via Attention Box Prediction and Aesthetics Assessment](https://mlsanity.com/paper/1710.08014/) (2017)
- [User Preferences Modeling and Learning for Pleasing Photo Collage Generation](https://mlsanity.com/paper/1506.00527/) (2015)

## Built on by (most cited first, 1 of 1)

- [Self-supervised Photographic Image Layout Representation Learning](https://mlsanity.com/paper/2403.03740/) (2024)

## Abstract (arXiv)

Photo collage aims to automatically arrange multiple photos on a given canvas with high aesthetic quality. Existing methods are based mainly on handcrafted feature optimization, which cannot adequately capture high-level human aesthetic senses. Deep learning provides a promising way, but owing to the complexity of collage and lack of training data, a solution has yet to be found. In this paper, we propose a novel pipeline for automatic generation of aspect ratio specified collage and the reinforcement learning technique is introduced in collage for the first time. Inspired by manual collages, we model the collage generation as sequential decision process to adjust spatial positions, orientation angles, placement order and the global layout. To instruct the agent to improve both the overall layout and local details, the reward function is specially designed for collage, considering subjective and objective factors. To overcome the lack of training data, we pretrain our deep aesthetic network on a large scale image aesthetic dataset (CPC) for general aesthetic feature extraction and propose an attention fusion module for structural collage feature representation. We test our model against competing methods on two movie datasets and our results outperform others in aesthetic quality evaluation. Further user study is also conducted to demonstrate the effectiveness.