# LLM-Driven Completeness and Consistency Evaluation for Cultural Heritage Data Augmentation in Cross-Modal Retrieval

arXiv: https://arxiv.org/abs/2511.06268 · HTML: https://mlsanity.com/paper/2511.06268/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 40 references that are in the map.
- First posted on arXiv 9 Nov 2025.

## Builds on (most cited first, 25 of 25)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling](https://mlsanity.com/paper/2501.17811/) (2025)
- [Stacked Cross Attention for Image-Text Matching](https://mlsanity.com/paper/1803.08024/) (2018)
- [Long-CLIP: Unlocking the Long-Text Capability of CLIP](https://mlsanity.com/paper/2403.15378/) (2024)
- [Improving CLIP Training with Language Rewrites](https://mlsanity.com/paper/2305.20088/) (2023)
- [Automatic Prompt Augmentation and Selection with Chain-of-Thought from Labeled Data](https://mlsanity.com/paper/2302.12822/) (2023)
- [Modeling Coverage for Neural Machine Translation](https://mlsanity.com/paper/1601.04811/) (2016)
- [Chinese CLIP: Contrastive Vision-Language Pretraining in Chinese](https://mlsanity.com/paper/2211.01335/) (2022)
- [LightningDOT: Pre-training Visual-Semantic Embeddings for Real-Time Image-Text Retrieval](https://mlsanity.com/paper/2103.08784/) (2021)
- [COTS: Collaborative Two-Stream Vision-Language Pre-Training Model for Cross-Modal Retrieval](https://mlsanity.com/paper/2204.07441/) (2022)
- [LoTLIP: Improving Language-Image Pre-training for Long Text Understanding](https://mlsanity.com/paper/2410.05249/) (2024)
- [LLMvsSmall Model? Large Language Model Based Text Augmentation Enhanced Personality Detection Model](https://mlsanity.com/paper/2403.07581/) (2024)
- [Improving Audio Captioning Models with Fine-grained Audio Features, Text Embedding Supervision, and LLM Mix-up Augmentation](https://mlsanity.com/paper/2309.17352/) (2023)
- [GOAL: Global-local Object Alignment Learning](https://mlsanity.com/paper/2503.17782/) (2025)
- [Time Travel: A Comprehensive Benchmark to Evaluate LMMs on Historical and Cultural Artifacts](https://mlsanity.com/paper/2502.14865/) (2025)
- [Enhancing Recipe Retrieval with Foundation Models: A Data Augmentation Perspective](https://mlsanity.com/paper/2312.04763/) (2023)
- [Unified Lexical Representation for Interpretable Visual-Language Alignment](https://mlsanity.com/paper/2407.17827/) (2024)
- [Multi-modal Reference Learning for Fine-grained Text-to-Image Retrieval](https://mlsanity.com/paper/2504.07718/) (2025)
- [Towards Cross-modal Retrieval in Chinese Cultural Heritage Documents: Dataset and Solution](https://mlsanity.com/paper/2505.10921/) (2025)
- [Flickr30K-CFQ: A Compact and Fragmented Query Dataset for Text-image Retrieval](https://mlsanity.com/paper/2403.13317/) (2024)

## Built on by (most cited first, 1 of 1)

- [AtomRec: Evolving Atomic Memory for Agentic Recommendation](https://mlsanity.com/paper/2609.04882/) (2026)

## Abstract (arXiv)

Cross-modal retrieval is essential for interpreting cultural heritage data, but its effectiveness is often limited by incomplete or inconsistent textual descriptions, caused by historical data loss and the high cost of expert annotation. While large language models (LLMs) offer a promising solution by enriching textual descriptions, their outputs frequently suffer from hallucinations or miss visually grounded details. To address these challenges, we propose C3, a data augmentation framework that enhances cross-modal retrieval performance by improving the completeness and consistency of LLM-generated descriptions. C3 introduces a completeness evaluation module to assess semantic coverage using both visual cues and language-model outputs. Furthermore, to mitigate factual inconsistencies, we formulate a Markov Decision Process to supervise Chain-of-Thought reasoning, guiding consistency evaluation through adaptive query control. Experiments on the cultural heritage datasets CulTi and TimeTravel, as well as on general benchmarks MSCOCO and Flickr30K, demonstrate that C3 achieves state-of-the-art performance in both fine-tuned and zero-shot settings.