# Language-Guided Visual Perception Disentanglement for Image Quality Assessment and Conditional Image Generation arXiv: https://arxiv.org/abs/2503.02206 ยท HTML: https://mlsanity.com/paper/2503.02206/ ## Key facts - Cited by 5 papers in the map (data through 14 Oct 2026). - Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. - Builds on 38 of its 66 references that are in the map. - First posted on arXiv 4 Mar 2025. ## Placed in - Multimodal LLMs / vision-language models / [Prompt learning and adaptation of CLIP](https://mlsanity.com/mllm/prompt-learning-and-adaptation-of-clip/) ## Builds on (most cited first, 30 of 38) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021) - [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023) - [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021) - [IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2308.06721/) (2023) - [Microsoft COCO Captions: Data Collection and Evaluation Server](https://mlsanity.com/paper/1504.00325/) (2015) - [MUSIQ: Multi-scale Image Quality Transformer](https://mlsanity.com/paper/2108.05997/) (2021) - [ShareGPT4V: Improving Large Multi-Modal Models with Better Captions](https://mlsanity.com/paper/2311.12793/) (2023) - [Exploring CLIP for Assessing the Look and Feel of Images](https://mlsanity.com/paper/2207.12396/) (2022) - [BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editing](https://mlsanity.com/paper/2305.14720/) (2023) - [NIMA: Neural Image Assessment](https://mlsanity.com/paper/1709.05424/) (2017) - [KonIQ-10k: An ecologically valid database for deep learning of blind image quality assessment](https://mlsanity.com/paper/1910.06180/) (2019) - [Blind Image Quality Assessment Using A Deep Bilinear Convolutional Neural Network](https://mlsanity.com/paper/1907.02665/) (2019) - [Diverse Image-to-Image Translation via Disentangled Representations](https://mlsanity.com/paper/1808.00948/) (2018) - [Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspective](https://mlsanity.com/paper/2303.14968/) (2023) - [Q-Bench: A Benchmark for General-Purpose Foundation Models on Low-level Vision](https://mlsanity.com/paper/2309.14181/) (2023) - [Massive Online Crowdsourced Study of Subjective and Objective Picture Quality](https://mlsanity.com/paper/1511.02919/) (2015) - [Versatile Diffusion: Text, Images and Variations All in One Diffusion Model](https://mlsanity.com/paper/2211.08332/) (2022) - [InstantStyle: Free Lunch towards Style-Preserving in Text-to-Image Generation](https://mlsanity.com/paper/2404.02733/) (2024) - [Q-Instruct: Improving Low-level Visual Abilities for Multi-modality Foundation Models](https://mlsanity.com/paper/2311.06783/) (2023) - [MetaIQA: Deep Meta-learning for No-Reference Image Quality Assessment](https://mlsanity.com/paper/2004.05508/) (2020) - [AGIQA-3K: An Open Database for AI-Generated Image Quality Assessment](https://mlsanity.com/paper/2306.04717/) (2023) - [Rich Human Feedback for Text-to-Image Generation](https://mlsanity.com/paper/2312.10240/) (2023) - [Photo Aesthetics Ranking Network with Attributes and Content Adaptation](https://mlsanity.com/paper/1606.01621/) (2016) - [Multimodal Contrastive Training for Visual Representation Learning](https://mlsanity.com/paper/2104.12836/) (2021) - [Iterative Prompt Learning for Unsupervised Backlit Image Enhancement](https://mlsanity.com/paper/2303.17569/) (2023) - [Controlling Perceptual Factors in Neural Style Transfer](https://mlsanity.com/paper/1611.07865/) (2016) - [VILA: Learning Image Aesthetics from User Comments with Vision-Language Pretraining](https://mlsanity.com/paper/2303.14302/) (2023) - [AesBench: An Expert Benchmark for Multimodal Large Language Models on Image Aesthetics Perception](https://mlsanity.com/paper/2401.08276/) (2024) - [Personalized Image Aesthetics Assessment with Rich Attributes](https://mlsanity.com/paper/2203.16754/) (2022) ## Built on by (most cited first, 5 of 5) - [Fine-grained Image Quality Assessment for Perceptual Image Restoration](https://mlsanity.com/paper/2508.14475/) (2025) - [LongT2IBench: A Benchmark for Evaluating Long Text-to-Image Generation with Graph-structured Annotations](https://mlsanity.com/paper/2512.09271/) (2025) - [Fine-grained Image Aesthetic Assessment: Learning Discriminative Scores from Relative Ranks](https://mlsanity.com/paper/2603.03907/) (2026) - [Omni-Attribute: Open-vocabulary Attribute Encoder for Visual Concept Personalization](https://mlsanity.com/paper/2512.10955/) (2025) - [Personalized Image Aesthetic Assessment via Preference-rich Sample Mining and Cohort Merging](https://mlsanity.com/paper/2607.15752/) (2026) ## Abstract (arXiv) Contrastive vision-language models, such as CLIP, have demonstrated excellent zero-shot capability across semantic recognition tasks, mainly attributed to the training on a large-scale I&1T (one Image with one Text) dataset. This kind of multimodal representations often blend semantic and perceptual elements, placing a particular emphasis on semantics. However, this could be problematic for popular tasks like image quality assessment (IQA) and conditional image generation (CIG), which typically need to have fine control on perceptual and semantic features. Motivated by the above facts, this paper presents a new multimodal disentangled representation learning framework, which leverages disentangled text to guide image disentanglement. To this end, we first build an I&2T (one Image with a perceptual Text and a semantic Text) dataset, which consists of disentangled perceptual and semantic text descriptions for an image. Then, the disentangled text descriptions are utilized as supervisory signals to disentangle pure perceptual representations from CLIP's original `coarse' feature space, dubbed DeCLIP. Finally, the decoupled feature representations are used for both image quality assessment (technical quality and aesthetic quality) and conditional image generation. Extensive experiments and comparisons have demonstrated the advantages of the proposed method on the two popular tasks. The dataset, code, and model will be available.