# Enhancing Image Aesthetics with Dual-Conditioned Diffusion Models Guided by Multimodal Perception

arXiv: https://arxiv.org/abs/2603.11556 · HTML: https://mlsanity.com/paper/2603.11556/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 21 of its 40 references that are in the map.
- First posted on arXiv 12 Mar 2026.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 21 of 21)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [DiffEdit: Diffusion-based semantic image editing with mask guidance](https://mlsanity.com/paper/2210.11427/) (2022)
- [Holistically-Nested Edge Detection](https://mlsanity.com/paper/1504.06375/) (2015)
- [Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levels](https://mlsanity.com/paper/2312.17090/) (2023)
- [NIMA: Neural Image Assessment](https://mlsanity.com/paper/1709.05424/) (2017)
- [KonIQ-10k: An ecologically valid database for deep learning of blind image quality assessment](https://mlsanity.com/paper/1910.06180/) (2019)
- [DragDiffusion: Harnessing Diffusion Models for Interactive Point-based Image Editing](https://mlsanity.com/paper/2306.14435/) (2023)
- [SmartBrush: Text and Shape Guided Object Inpainting with Diffusion Model](https://mlsanity.com/paper/2212.05034/) (2022)
- [End-to-End Diffusion Latent Optimization Improves Classifier Guidance](https://mlsanity.com/paper/2303.13703/) (2023)
- [Depicting Beyond Scores: Advancing Image Quality Assessment through Multi-modal Language Models](https://mlsanity.com/paper/2312.08962/) (2023)
- [StyleDiffusion: Prompt-Embedding Inversion for Text-Based Editing](https://mlsanity.com/paper/2303.15649/) (2023)
- [Image Aesthetic Assessment: An Experimental Survey](https://mlsanity.com/paper/1610.00838/) (2016)
- [UNIAA: A Unified Multi-modal Image Aesthetic Assessment Baseline and Benchmark](https://mlsanity.com/paper/2404.09619/) (2024)
- [Aesthetic-Driven Image Enhancement by Adversarial Learning](https://mlsanity.com/paper/1707.05251/) (2017)
- [Quality Assessment of Image Super-Resolution: Balancing Deterministic and Statistical Fidelity](https://mlsanity.com/paper/2207.08689/) (2022)
- [RAD: Region-Aware Diffusion Models for Image Inpainting](https://mlsanity.com/paper/2412.09191/) (2024)
- [One-Shot Structure-Aware Stylized Image Synthesis](https://mlsanity.com/paper/2402.17275/) (2024)
- [NICER: Aesthetic Image Enhancement with Humans in the Loop](https://mlsanity.com/paper/2012.01778/) (2020)

## Abstract (arXiv)

Image aesthetic enhancement aims to perceive aesthetic deficiencies in images and perform corresponding editing operations, which is highly challenging and requires the model to possess creativity and aesthetic perception capabilities. Although recent advancements in image editing models have significantly enhanced their controllability and flexibility, they struggle with enhancing image aesthetic. The primary challenges are twofold: first, following editing instructions with aesthetic perception is difficult, and second, there is a scarcity of "perfectly-paired" images that have consistent content but distinct aesthetic qualities. In this paper, we propose Dual-supervised Image Aesthetic Enhancement (DIAE), a diffusion-based generative model with multimodal aesthetic perception. First, DIAE incorporates Multimodal Aesthetic Perception (MAP) to convert the ambiguous aesthetic instruction into explicit guidance by (i) employing detailed, standardized aesthetic instructions across multiple aesthetic attributes, and (ii) utilizing multimodal control signals derived from text-image pairs that maintain consistency within the same aesthetic attribute. Second, to mitigate the lack of "perfectly-paired" images, we collect "imperfectly-paired" dataset called IIAEData, consisting of images with varying aesthetic qualities while sharing identical semantics. To better leverage the weak matching characteristics of IIAEData during training, a dual-branch supervision framework is also introduced for weakly supervised image aesthetic enhancement. Experimental results demonstrate that DIAE outperforms the baselines and obtains superior image aesthetic scores and image content consistency scores.