mlsanityData to 13 Oct 2026

Context-Infused Visual Grounding for Art

arXiv 2410.12369 ↗First posted 16 Oct 2024cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

Many artwork collections contain textual attributes that provide rich and contextualised descriptions of artworks. Visual grounding offers the potential for localising subjects within these descriptions on images, however, existing approaches are trained on natural images and generalise poorly to art. In this paper, we present CIGAr (Context-Infused GroundingDINO for Art), a visual grounding approach which utilises the artwork descriptions during training as context, thereby enabling visual grounding on art. In addition, we present a new dataset, Ukiyo-eVG, with manually annotated phrase-grounding annotations, and we set a new state-of-the-art for object detection on two artwork datasets.

Key facts. Cited by 2 papers in the map (data through 13 Oct 2026). Builds on 31 of its 65 references that are in the map. First posted on arXiv 16 Oct 2024.

Builds on

31 of 65 references.

References · 31

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,304
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,588
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,550
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,549
  5. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,855
  6. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,299
  7. 2016Modeling Context in Referring Expressionscited by 1,057
  8. 2021Open-vocabulary Object Detection via Vision and Language Knowledge Distillationcited by 903
  9. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 898
  10. 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 815
  11. 2018Cross-Domain Weakly-Supervised Object Detection through Progressive Domain Adaptationcited by 218
  12. 2015Large-scale Classification of Fine-Art Paintings: Learning The Right Metric on The Right Featurecited by 182
19 more
  1. 2022DetCLIP: Dictionary-Enriched Visual-Concept Paralleled Pre-training for Open-world Detectioncited by 176
  2. 2015Learning Deep Structure-Preserving Image-Text Embeddingscited by 139
  3. 2020Linguistic Structure Guided Context Modeling for Referring Image Segmentationcited by 102
  4. 2018How to Read Paintings: Semantic Art Understanding with Multi-Modal Retrievalcited by 74
  5. 2019CLEVR-Ref+: Diagnosing Visual Reasoning with Referring Expressionscited by 73
  6. 2022Improving Visual Grounding with Visual-Linguistic Verification and Iterative Reasoningcited by 69
  7. 2020Graph-Structured Referring Expression Reasoning in The Wildcited by 58
  8. 2018Weakly Supervised Object Detection in Artworkscited by 27
  9. 2016Detecting People in Artwork with CNNscited by 25
  10. 2020KaoKore: A Pre-modern Japanese Art Facial Expression Datasetcited by 18
  11. 2020A Data Set and a Convolutional Model for Iconography Classification in Paintingscited by 17
  12. 2023Advancing Visual Grounding with Scene Knowledge: Benchmark and Methodcited by 17
  13. 2017OmniArt: Multi-task Deep Learning for Artistic Data Analysiscited by 16
  14. 2019Context-Aware Embeddings for Automatic Art Analysiscited by 16
  15. 2021Iconographic Image Captioning for Artworkscited by 15
  16. 2021Improving Object Detection in Art Images Using Only Style Transfercited by 15
  17. 2020Multiple instance learning on deep features for weakly supervised object detection with extreme domain shiftscited by 10
  18. 2017Aligned Image-Word Representations Improve Inductive Transfer Across Vision-Language Taskscited by 9
  19. 2022Incremental Object Grounding Using Scene Graphscited by 6

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 1.

2 papers.

  1. 2024No Annotations for Object Detection in Art through Stable Diffusioncited by 2
  2. 2025"Jutters"cited by 0