# Large Multi-modality Model Assisted AI-Generated Image Quality Assessment

arXiv: https://arxiv.org/abs/2404.17762 · HTML: https://mlsanity.com/paper/2404.17762/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 56 references that are in the map.
- First posted on arXiv 27 Apr 2024.

## Builds on (most cited first, 30 of 32)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [MUSIQ: Multi-scale Image Quality Transformer](https://mlsanity.com/paper/2108.05997/) (2021)
- [Exploring CLIP for Assessing the Look and Feel of Images](https://mlsanity.com/paper/2207.12396/) (2022)
- [Training Diffusion Models with Reinforcement Learning](https://mlsanity.com/paper/2305.13301/) (2023)
- [PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis](https://mlsanity.com/paper/2310.00426/) (2023)
- [AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks](https://mlsanity.com/paper/1711.10485/) (2017)
- [LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Model](https://mlsanity.com/paper/2304.15010/) (2023)
- [mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaboration](https://mlsanity.com/paper/2311.04257/) (2023)
- [KonIQ-10k: An ecologically valid database for deep learning of blind image quality assessment](https://mlsanity.com/paper/1910.06180/) (2019)
- [Blind Image Quality Assessment Using A Deep Bilinear Convolutional Neural Network](https://mlsanity.com/paper/1907.02665/) (2019)
- [Q-Bench: A Benchmark for General-Purpose Foundation Models on Low-level Vision](https://mlsanity.com/paper/2309.14181/) (2023)
- [Multimodal Foundation Models: From Specialists to General-Purpose Assistants](https://mlsanity.com/paper/2309.10020/) (2023)
- [LCM-LoRA: A Universal Stable-Diffusion Acceleration Module](https://mlsanity.com/paper/2311.05556/) (2023)
- [No-Reference Image Quality Assessment via Transformers, Relative Ranking, and Self-Consistency](https://mlsanity.com/paper/2108.06858/) (2021)
- [Q-Instruct: Improving Low-level Visual Abilities for Multi-modality Foundation Models](https://mlsanity.com/paper/2311.06783/) (2023)
- [Towards Open-ended Visual Quality Comparison](https://mlsanity.com/paper/2402.16641/) (2024)
- [Transformer for Image Quality Assessment](https://mlsanity.com/paper/2101.01097/) (2020)
- [Blind Quality Assessment for in-the-Wild Images via Hierarchical Feature Fusion and Iterative Mixed Database Training](https://mlsanity.com/paper/2105.14550/) (2021)
- [Text-Guided Synthesis of Artistic Images with Retrieval-Augmented Diffusion Models](https://mlsanity.com/paper/2207.13038/) (2022)
- [Progressive Knowledge Distillation Of Stable Diffusion XL Using Layer Level Loss](https://mlsanity.com/paper/2401.02677/) (2024)
- [Towards Artistic Image Aesthetics Assessment: a Large-scale Dataset and a New Method](https://mlsanity.com/paper/2303.15166/) (2023)

## Built on by (most cited first, 24 of 24)

- [Breaking the Modality Barrier: Universal Embedding Learning with Multimodal LLMs](https://mlsanity.com/paper/2504.17432/) (2025)
- [LMM4Edit: Benchmarking and Evaluating Multimodal Image Editing with LMMs](https://mlsanity.com/paper/2507.16193/) (2025)
- [AGHI-QA: A Subjective-Aligned Dataset and Metric for AI-Generated Human Images](https://mlsanity.com/paper/2504.21308/) (2025)
- [Human-Activity AGV Quality Assessment: A Benchmark Dataset and an Objective Evaluation Metric](https://mlsanity.com/paper/2411.16619/) (2024)
- [HarmonyIQA: Pioneering Benchmark and Model for Image Harmonization Quality Assessment](https://mlsanity.com/paper/2501.01116/) (2025)
- [Text-Visual Semantic Constrained AI-Generated Image Quality Assessment](https://mlsanity.com/paper/2507.10432/) (2025)
- [Engagement Prediction of Short Videos with Large Multimodal Models](https://mlsanity.com/paper/2508.02516/) (2025)
- [EditHF-1M: A Million-Scale Rich Human Preference Feedback for Image Editing](https://mlsanity.com/paper/2603.14916/) (2026)
- [Decoupling Semantics from Distortions: Multi-Scale Two-Stream Vision-Language Alignment for AI-Generated Image Quality Assessment](https://mlsanity.com/paper/2606.16799/) (2026)
- [Quality Prediction of AI Generated Images and Videos: Emerging Trends and Opportunities](https://mlsanity.com/paper/2410.08534/) (2024)
- [Efficient Face Image Quality Assessment via Self-training and Knowledge Distillation](https://mlsanity.com/paper/2507.15709/) (2025)
- [M3-AGIQA: Multimodal, Multi-Round, Multi-Aspect AI-Generated Image Quality Assessment](https://mlsanity.com/paper/2502.15167/) (2025)
- [Multi-Dimensional Quality Assessment for Text-to-3D Assets: Dataset and Model](https://mlsanity.com/paper/2502.16915/) (2025)
- [Towards Explainable Partial-AIGC Image Quality Assessment](https://mlsanity.com/paper/2504.09291/) (2025)
- [Multi-Dimensional Quality Assessment for AI-Generated Human-Centric Videos: Dataset and Model](https://mlsanity.com/paper/2607.16742/) (2026)
- [Decoupling Perception and Calibration: Label-Efficient Image Quality Assessment Framework](https://mlsanity.com/paper/2601.20689/) (2026)
- [ELIQ: A Label-Free Framework for Quality Assessment of Evolving AI-Generated Images](https://mlsanity.com/paper/2602.03558/) (2026)
- [Tool-IQA: Augmenting Image Quality Assessment with Simple Tools](https://mlsanity.com/paper/2606.16082/) (2026)
- [CAS-IQA: Teaching Vision-Language Models for Synthetic Angiography Quality Assessment](https://mlsanity.com/paper/2505.17619/) (2025)
- [R4-CGQA: Retrieval-based Vision Language Models for Computer Graphics Image Quality Assessment](https://mlsanity.com/paper/2603.10578/) (2026)
- [Redefining Quality Criteria and Distance-Aware Score Modeling for Image Editing Assessment](https://mlsanity.com/paper/2604.12175/) (2026)
- [LEIQ-Assessor: Multi-dimensional Quality Assessment of Low-light Enhanced Images via Multi-task Learning](https://mlsanity.com/paper/2606.29752/) (2026)
- [DroneIQA-VLE: Multi-Task Drone Image Quality Assessment via Vision-Language Ensemble](https://mlsanity.com/paper/2607.00416/) (2026)
- [Bridging Adversarial and Collaborative Learning for AI-Generated Image Quality Assessment](https://mlsanity.com/paper/2608.24372/) (2026)

## Abstract (arXiv)

Traditional deep neural network (DNN)-based image quality assessment (IQA) models leverage convolutional neural networks (CNN) or Transformer to learn the quality-aware feature representation, achieving commendable performance on natural scene images. However, when applied to AI-Generated images (AGIs), these DNN-based IQA models exhibit subpar performance. This situation is largely due to the semantic inaccuracies inherent in certain AGIs caused by uncontrollable nature of the generation process. Thus, the capability to discern semantic content becomes crucial for assessing the quality of AGIs. Traditional DNN-based IQA models, constrained by limited parameter complexity and training data, struggle to capture complex fine-grained semantic features, making it challenging to grasp the existence and coherence of semantic content of the entire image. To address the shortfall in semantic content perception of current IQA models, we introduce a large Multi-modality model Assisted AI-Generated Image Quality Assessment (MA-AGIQA) model, which utilizes semantically informed guidance to sense semantic information and extract semantic vectors through carefully designed text prompts. Moreover, it employs a mixture of experts (MoE) structure to dynamically integrate the semantic information with the quality-aware features extracted by traditional DNN-based IQA models. Comprehensive experiments conducted on two AI-generated content datasets, AIGCQA-20k and AGIQA-3k show that MA-AGIQA achieves state-of-the-art performance, and demonstrate its superior generalization capabilities on assessing the quality of AGIs. Code is available at https://github.com/wangpuyi/MA-AGIQA.