# On Semiotic-Grounded Interpretive Evaluation of Generative Art

arXiv: https://arxiv.org/abs/2604.08641 · HTML: https://mlsanity.com/paper/2604.08641/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 79 references that are in the map.
- First posted on arXiv 9 Apr 2026.

## Builds on (most cited first, 20 of 20)

- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Qwen-Image Technical Report](https://mlsanity.com/paper/2508.02324/) (2025)
- [Exploring CLIP for Assessing the Look and Feel of Images](https://mlsanity.com/paper/2207.12396/) (2022)
- [Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation](https://mlsanity.com/paper/2305.01569/) (2023)
- [TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answering](https://mlsanity.com/paper/2303.11897/) (2023)
- [Human Preference Score: Better Aligning Text-to-Image Models with Human Preference](https://mlsanity.com/paper/2303.14420/) (2023)
- [VIEScore: Towards Explainable Metrics for Conditional Image Synthesis Evaluation](https://mlsanity.com/paper/2312.14867/) (2023)
- [How to Read Paintings: Semantic Art Understanding with Multi-Modal Retrieval](https://mlsanity.com/paper/1810.09617/) (2018)
- [ArtiMuse: Fine-Grained Image Aesthetics Assessment with Joint Scoring and Expert-Level Understanding](https://mlsanity.com/paper/2507.14533/) (2025)
- [LMM4LMM: Benchmarking and Evaluating Large-multimodal Image Generation with LMMs](https://mlsanity.com/paper/2504.08358/) (2025)
- [The Prompt Artists](https://mlsanity.com/paper/2303.12253/) (2023)
- [Organic or Diffused: Can We Distinguish Human Art from AI-generated Images?](https://mlsanity.com/paper/2402.03214/) (2024)
- [Can Multimodal Large Language Models Understand Spatial Relations?](https://mlsanity.com/paper/2505.19015/) (2025)
- [Forgotten Polygons: Multimodal Large Language Models are Shape-Blind](https://mlsanity.com/paper/2502.15969/) (2025)
- [Multimodal LLMs Can Reason about Aesthetics in Zero-Shot](https://mlsanity.com/paper/2501.09012/) (2025)
- [ArtWhisperer: A Dataset for Characterizing Human-AI Interactions in Artistic Creations](https://mlsanity.com/paper/2306.08141/) (2023)
- [ArtRAG: Retrieval-Augmented Generation with Structured Context for Visual Art Understanding](https://mlsanity.com/paper/2505.06020/) (2025)
- [Not Minds, but Signs: Reframing LLMs through Semiotics](https://mlsanity.com/paper/2505.17080/) (2025)
- [For a semiotic AI: Bridging computer vision and visual semiotics for computational observation of large scale facial image archives](https://mlsanity.com/paper/2407.03268/) (2024)

## Abstract (arXiv)

Interpretation is essential to deciphering the language of art: audiences communicate with artists by recovering meaning from visual artifacts. However, current Generative Art (GenArt) evaluators remain fixated on surface-level image quality or literal prompt adherence, failing to assess the deeper symbolic or abstract meaning intended by the creator. We address this gap by formalizing a Peircean computational semiotic theory that models Human-GenArt Interaction (HGI) as cascaded semiosis. This framework reveals that artistic meaning is conveyed through three modes - iconic, symbolic, and indexical - yet existing evaluators operate heavily within the iconic mode, remaining structurally blind to the latter two. To overcome this structural blindness, we propose SemJudge. This evaluator explicitly assesses symbolic and indexical meaning in HGI via a Hierarchical Semiosis Graph (HSG) that reconstructs the meaning-making process from prompt to generated artifact. Extensive quantitative experiments show that SemJudge aligns more closely with human judgments than prior evaluators on an interpretation-intensive fine-art benchmark. User studies further demonstrate that SemJudge produces deeper, more insightful artistic interpretations, thereby paving the way for GenArt to move beyond the generation of "pretty" images toward a medium capable of expressing complex human experience. Project page: https://github.com/songrise/SemJudge.