mlsanityData to 14 Oct 2026

ViMRHP: A Vietnamese Benchmark Dataset for Multimodal Review Helpfulness Prediction via Human-AI Collaborative Annotation

arXiv 2505.07416 ↗First posted 12 May 2025cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Multimodal Review Helpfulness Prediction (MRHP) is an essential task in recommender systems, particularly in E-commerce platforms. Determining the helpfulness of user-generated reviews enhances user experience and improves consumer decision-making. However, existing datasets focus predominantly on English and Indonesian, resulting in a lack of linguistic diversity, especially for low-resource languages such as Vietnamese. In this paper, we introduce ViMRHP (Vietnamese Multimodal Review Helpfulness Prediction), a large-scale benchmark dataset for MRHP task in Vietnamese. This dataset covers four domains, including 2K products with 46K reviews. Meanwhile, a large-scale dataset requires considerable time and cost. To optimize the annotation process, we leverage AI to assist annotators in constructing the ViMRHP dataset. With AI assistance, annotation time is reduced (90 to 120 seconds per task down to 20 to 40 seconds per task) while maintaining data quality and lowering overall costs by approximately 65%. However, AI-generated annotations still have limitations in complex annotation tasks, which we further examine through a detailed performance analysis. In our experiment on ViMRHP, we evaluate baseline models on human-verified and AI-generated annotations to assess their quality differences. The ViMRHP dataset is publicly available at https://github.com/trng28/ViMRHP

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 16 of its 30 references that are in the map. First posted on arXiv 12 May 2025.

Builds on

16 of 30 references.

References · 16

  1. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,312
  2. 2024MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmarkcited by 372
  3. 2022Is GPT-3 a Good Data Annotator?cited by 208
  4. 2024Large Language Models for Data Annotation and Synthesis: A Surveycited by 205
  5. 2021Want To Reduce Labeling Cost? GPT-3 Can Helpcited by 196
  6. 2023AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotatorscited by 172
  7. 2017Bilateral Multi-Perspective Matching for Natural Language Sentencescited by 121
  8. 2023CoAnnotating: Uncertainty-Guided Work Allocation between Human and Large Language Models for Data Annotationcited by 94
  9. 2016Learning to Match Using Local and Distributed Representations of Text for Web Searchcited by 68
  10. 2016Text Matching as Image Recognitioncited by 59
  11. 2023Value FULCRA: Mapping Large Language Models to the Multidimensional Spectrum of Basic Human Valuescited by 50
  12. 2019MatchZoo: A Learning, Practicing, and Developing System for Neural Text Matchingcited by 17
4 more
  1. 2022Adaptive Contrastive Learning on Multimodal Transformer for Review Helpfulness Predictionscited by 13
  2. 2022SANCL: Multimodal Review Helpfulness Prediction with Selective Attention and Natural Contrastive Learningcited by 12
  3. 2024VIVA: A Benchmark for Vision-Grounded Decision-Making with Human Valuescited by 12
  4. 2023On the Role of Reviewer Expertise in Temporal Review Helpfulness Predictioncited by 6

Built on by

Not cited yet.