mlsanityData to 7 Oct 2026

COCO-CN for Cross-Lingual Image Tagging, Captioning and Retrieval

arXiv 1805.08661 ↗First posted 22 May 2018cs.CL cs.CVcited by 80

Not on the map. Why?

Cited by 23 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

This paper contributes to cross-lingual image annotation and retrieval in terms of data and baseline methods. We propose COCO-CN, a novel dataset enriching MS-COCO with manually written Chinese sentences and tags. For more effective annotation acquisition, we develop a recommendation-assisted collective annotation system, automatically providing an annotator with several tags and sentences deemed to be relevant with respect to the pictorial content. Having 20,342 images annotated with 27,218 Chinese sentences and 70,993 tags, COCO-CN is currently the largest Chinese-English dataset that provides a unified and challenging platform for cross-lingual image tagging, captioning and retrieval. We develop conceptually simple yet effective methods per task for learning from cross-lingual resources. Extensive experiments on the three tasks justify the viability of the proposed dataset and methods. Data and code are publicly available at https://github.com/li-xirong/coco-cn

Key facts. Cited by 80 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 May 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 14; 2022: 14; 2023: 18; 2024: 15; 2025: 14; 2026: 5.

80 papers.

Sort:

By area:

  1. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  2. 2021WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learningcited by 285
  3. 2021Visually Grounded Reasoning across Languages and Culturescited by 188
  4. 2021From Show to Tell: A Survey on Deep Learning-based Image Captioningcited by 137
  5. 2024AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmarkcited by 137
  6. 2023M3IT: A Large-Scale Dataset towards Multi-Modal Multilingual Instruction Tuningcited by 126
  7. 2022Wukong: A 100 Million Large-scale Chinese Cross-modal Pre-training Benchmarkcited by 114
  8. 2022Chinese CLIP: Contrastive Vision-Language Pretraining in Chinesecited by 108
  9. 2022AltCLIP: Altering the Language Encoder in CLIP for Extended Language Capabilitiescited by 84
  10. 2021UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-trainingcited by 64
  11. 2022IGLUE: A Benchmark for Transfer Learning across Modalities, Tasks, and Languagescited by 54
  12. 2021LightningDOT: Pre-training Visual-Semantic Embeddings for Real-Time Image-Text Retrievalcited by 44
38 more
  1. 2021MURAL: Multimodal, Multitask Retrieval Across Languagescited by 44
  2. 2025Ask in Any Modality: A Comprehensive Survey on Multimodal Retrieval-Augmented Generationcited by 42
  3. 2021ERNIE-ViLG: Unified Generative Pre-training for Bidirectional Vision-Language Generationcited by 41
  4. 2023mBLIP: Efficient Bootstrapping of Multilingual Vision-LLMscited by 41
  5. 2022Cross-View Language Modeling: Towards Unified Cross-Lingual Cross-Modal Pre-trainingcited by 24
  6. 2023NLLB-CLIP -- train performant multilingual image retrieval model on a budgetcited by 24
  7. 2022CCMB: A Large-scale Chinese Cross-modal Benchmarkcited by 21
  8. 2024No Culture Left Behind: ArtELingo-28, a Benchmark of WikiArt with Captions in 28 Languagescited by 19
  9. 2021GEM: A General Evaluation Benchmark for Multimodal Taskscited by 16
  10. 2023Cross2StrA: Unpaired Cross-lingual Image Captioning with Cross-lingual Cross-modal Structure-pivoted Alignmentcited by 15
  11. 2023Universal Multimodal Representation for Language Understandingcited by 14
  12. 2024Multilingual Diversity Improves Vision-Language Representationscited by 14
  13. 2024Multimodal LLM Enhanced Cross-lingual Cross-modal Retrievalcited by 13
  14. 2024Incorporating Visual Experts to Resolve the Information Loss in Multimodal Large Language Modelscited by 12
  15. 2023Babel-ImageNet: Massively Multilingual Evaluation of Vision-and-Language Representationscited by 11
  16. 2024An Empirical Study and Analysis of Text-to-Image Generation Using Large Language Model-Powered Textual Representationcited by 11
  17. 2024LLM2CLIP: Powerful Language Model Unlocks Richer Cross-Modality Representationcited by 11
  18. 2025FG-CLIP 2: A Bilingual Fine-grained Vision-Language Alignment Modelcited by 11
  19. 2023CL2CM: Improving Cross-Lingual Cross-Modal Retrieval via Cross-Lingual Knowledge Transfercited by 10
  20. 2024Taiyi-Diffusion-XL: Advancing Bilingual Text-to-Image Generation with Large Vision-Language Model Supportcited by 10
  21. 2024MATE: Meet At The Embedding -- Connecting Images with Long Textscited by 10
  22. 2021Answer Questions with Right Image Regions: A Visual Attention Regularization Approachcited by 9
  23. 2021Object-Centric Unsupervised Image Captioningcited by 9
  24. 2022MaXM: Towards Multilingual Visual Question Answeringcited by 9
  25. 2022X2-VLM: All-In-One Pre-trained Model For Vision-Language Taskscited by 9
  26. 2023Translation-Enhanced Multilingual Text-to-Image Generationcited by 9
  27. 2023Ziya-Visual: Bilingual Large Vision-Language Model via Multi-Task Instruction Tuningcited by 9
  28. 2025AuroraLong: Bringing RNNs Back to Efficient Open-Ended Video Understandingcited by 9
  29. 2021MTVR: Multilingual Moment Retrieval in Videoscited by 8
  30. 2022ERNIE-ViL 2.0: Multi-view Contrastive Learning for Image-Text Pre-trainingcited by 8
  31. 2023Semantic and Expressive Variation in Image Captions Across Languagescited by 8
  32. 2024CVLUE: A New Benchmark Dataset for Chinese Vision-Language Understanding Evaluationcited by 8
  33. 2025OpenEvents V1: Large-Scale Benchmark Dataset for Multimodal Event Groundingcited by 8
  34. 2022Image Captioning via Compact Bidirectional Architecturecited by 7
  35. 2022ERNIE-UniX2: A Unified Cross-lingual Cross-modal Framework for Understanding and Generationcited by 7
  36. 2021Towards Zero-shot Cross-lingual Image Retrieval and Taggingcited by 4
  37. 2021FooDI-ML: a large multi-language dataset of food, drinks and groceries images and descriptionscited by 4
  38. 2023Efficient Cross-Lingual Transfer for Chinese Stable Diffusion with Images as Pivotscited by 4

50 most cited of 80 All 80 (JSON)