# Describe-to-Score: A text-guided framework for image complexity assessment

arXiv: https://arxiv.org/abs/2509.16609 · HTML: https://mlsanity.com/paper/2509.16609/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 53 references that are in the map.
- First posted on arXiv 20 Sep 2025.

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [TOPIQ: A Top-down Approach from Semantics to Distortions for Image Quality Assessment](https://mlsanity.com/paper/2308.03060/) (2023)
- [From Patches to Pictures (PaQ-2-PiQ): Mapping the Perceptual Space of Picture Quality](https://mlsanity.com/paper/1912.10088/) (2019)
- [Convolutions Die Hard: Open-Vocabulary Segmentation with Single Frozen Convolutional CLIP](https://mlsanity.com/paper/2308.02487/) (2023)
- [ARNIQA: Learning Distortion Manifold for Image Quality Assessment](https://mlsanity.com/paper/2310.14918/) (2023)
- [Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Survey](https://mlsanity.com/paper/2411.17558/) (2024)
- [Cross-Modal Retrieval: A Systematic Review of Methods and Future Directions](https://mlsanity.com/paper/2308.14263/) (2023)
- [CLICv2: Image Complexity Representation via Content Invariance Contrastive Learning](https://mlsanity.com/paper/2503.06641/) (2025)
- [Content-Distortion High-Order Interaction for Blind Image Quality Assessment](https://mlsanity.com/paper/2504.05076/) (2025)
- [DGIQA: Depth-guided Feature Attention and Refinement for Generalizable Image Quality Assessment](https://mlsanity.com/paper/2505.24002/) (2025)
- [Segmenting and Understanding: Region-aware Semantic Attention for Fine-grained Image Quality Assessment with Large Language Models](https://mlsanity.com/paper/2508.07818/) (2025)
- [Attention Down-Sampling Transformer, Relative Ranking and Self-Consistency for Blind Image Quality Assessment](https://mlsanity.com/paper/2409.07115/) (2024)

## Abstract (arXiv)

Accurately assessing image complexity (IC) is essential for many vision tasks, yet existing approaches rely almost exclusively on visual features and therefore fail to capture the high-level semantics that humans often use when judging complexity. We introduce a multimodal perspective for IC modeling by integrating visual representations with caption-derived textual semantics. This integration enriches the representational space and provides complementary structural cues that are difficult to infer from vision alone. From an information theoretic and representation viewpoint, we offer an idealized analysis suggesting how semantic guidance can regularize the hypothesis space and support more stable generalization. We propose D2S (Describe-to-Score), a text-guided framework that uses caption-derived semantics only during training to regularize visual complexity modeling, while preserving a vision-only inference pipeline with no additional multimodal overhead at inference. Concretely, D2S transfers semantic structure into the visual branch through feature alignment and entropy distribution alignment, encouraging the visual encoder to internalize complexity-relevant semantic regularities. Experiments show that D2S achieves state-of-the-art performance on the IC9600 benchmark and remains competitive on no-reference image quality assessment (NR-IQA) tasks. Additional analyses further clarify the sample-imbalance issue in the small samples training setting and the distribution-shift limitations observed in cross-dataset transfer. Code is available at: https://github.com/xauat-liushipeng/D2S.