mlsanityData to 14 Oct 2026

ArtAnno: Annotating Implicit Semantics in Artworks through LLM Agent-Driven Bidirectional Human-AI Augmentation

arXiv 2608.05026 ↗First posted 5 Aug 2026cs.HC cs.AIcited by 0

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

High-quality annotation of artworks is essential for computational art research, yet extracting implicit semantics remains challenging due to the reliance on culturally grounded meanings and deep contextual knowledge behind the images. Current AI-assisted annotation tools often lack assistance or rely on one-way workflows where experts have to perform extra manual calibrations to improve AI models, resulting in limited efficiency. To address this, we propose Bidirectional Human-AI Augmentation(BiHAA), a closed-loop framework in which skills and domain knowledge base evolve through real-time interaction and bidirectional HAI augmentation. Informed by a formative study with 20 artwork annotators from different backgrounds, we implement this framework in ArtAnno, an artwork annotation system driven by a multi-agent architecture. The system includes a Proactive Agentic Support Module, where AI augments humans through semantic mining and label suggestion, and an Interaction-Driven Evolution Module, where human expertise continuously enhances the AI through distilling annotation trajectories into reusable experience. Evaluation through a user study and two case studies demonstrates that our framework and system improve annotation efficiency, enable knowledge accumulation, and reduce the effort of information seeking and verification for annotators with limited domain expertise. We conclude by discussing broader implications and future directions.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 23 of its 61 references that are in the map. First posted on arXiv 5 Aug 2026.

Builds on

23 of 61 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024VisionGPT: Vision-Language Understanding Agent Using Generalized Multimodal Framework14 Mar

Other references · 22

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  2. 2024YOLO-World: Real-Time Open-Vocabulary Object Detectioncited by 503
  3. 2026AutoSkill: Experience-Driven Lifelong Learning via Skill Self-Evolutioncited by 115
  4. 2021ArtEmis: Affective Language for Visual Artcited by 104
  5. 2025Understanding Software Engineering Agents: A Study of Thought-Action-Result Trajectoriescited by 81
  6. 2026XSkill: Continual Learning from Experience and Skills in Multimodal Agentscited by 75
  7. 2023CoQuest: Exploring Research Question Co-Creation with an LLM-based Agentcited by 34
  8. 2024DeepEdit: Knowledge Editing as Decoding with Constraintscited by 33
  9. 2025SciSciGPT: Advancing Human-AI Collaboration in the Science of Sciencecited by 27
  10. 2025LLaVA-4D: Embedding SpatioTemporal Prompt into LMMs for 4D Scene Understandingcited by 17
  11. 2025Human-LLM Coevolution: Evidence from Academic Writingcited by 15
  12. 2020Semi-Automatic Data Annotation guided by Feature Space Projectioncited by 11
10 more
  1. 2022DEArt: Dataset of European Artcited by 9
  2. 2024Modeling Collaborator: Enabling Subjective Vision Classification With Minimal Human Effort via LLM Tool-Usecited by 9
  3. 2025PosterMate: Audience-driven Collaborative Persona Agents for Poster Designcited by 9
  4. 2025ArtSeek: Deep artwork understanding via multimodal in-context reasoning and late interaction retrievalcited by 5
  5. 2025CrowdAgent: Multi-Agent Managed Multi-Source Annotation Systemcited by 5
  6. 2025VISTA: A Visual Analytics Framework to Enhance Foundation Model-Generated Data Labelscited by 3
  7. 2025Interaction-Augmented Instruction: Modeling the Synergy of Prompts and Interactions in Human-GenAI Collaborationcited by 1
  8. 2025Knowledge Graph for Intelligent Generation of Artistic Image Creation: Constructing a New Annotation Hierarchycited by 1
  9. 2026DuoDrama: Supporting Screenplay Refinement Through LLM-Assisted Human Reflectioncited by 1
  10. 2026Figures as Interfaces: Toward LLM-Native Artifacts for Scientific Discoverycited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026A-MAR: Agent-based Multimodal Art Retrieval for Fine-Grained Artwork Understanding
  2. 2026Sanyu Studio: A Multi-Agent System for Art-Historical Narrative Construction
  3. 2025ArtRAG: Retrieval-Augmented Generation with Structured Context for Visual Art Understanding
  4. 2025Context-aware Multimodal AI Reveals Hidden Pathways in Five Centuries of Art Evolution
  5. 2026M-ArtAgent: Evidence-Based Multimodal Agent for Implicit Art Influence Discovery
  6. 2026ArtSociety: Multi-Agent Multimodal Collaboration for Art Emotion Understanding
  7. 2026Conducting Stylistic Analysis of Paintings through an Art-History Agent
  8. 2025ArtMentor: AI-Assisted Evaluation of Artworks to Explore Multimodal Large Language Models Capabilities