mlsanityData to 14 Oct 2026

Leveraging Customer Feedback for Multi-modal Insight Extraction

arXiv 2410.09999 ↗First posted 13 Oct 2024cs.CL cs.AI cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Businesses can benefit from customer feedback in different modalities, such as text and images, to enhance their products and services. However, it is difficult to extract actionable and relevant pairs of text segments and images from customer feedback in a single pass. In this paper, we propose a novel multi-modal method that fuses image and text information in a latent space and decodes it to extract the relevant feedback segments using an image-text grounded text decoder. We also introduce a weakly-supervised data generation technique that produces training data for this task. We evaluate our model on unseen data and demonstrate that it can effectively mine actionable insights from multi-modal customer feedback, outperforming the existing baselines by 14 points in F1 score.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 26 of its 36 references that are in the map. First posted on arXiv 13 Oct 2024.

Builds on

26 of 36 references.

References · 26

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  6. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  7. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  8. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  9. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  10. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  11. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
  12. 2015VQA: Visual Question Answeringcited by 2,506
14 more
  1. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  2. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
  3. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,619
  4. 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,296
  5. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,230
  6. 2019Unified Vision-Language Pre-Training for Image Captioning and VQAcited by 494
  7. 2019Unicoder-VL: A Universal Encoder for Vision and Language by Cross-modal Pre-trainingcited by 451
  8. 2021VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Expertscited by 425
  9. 2021Unifying Vision-and-Language Tasks via Text Generationcited by 412
  10. 2019MASS: Masked Sequence to Sequence Pre-training for Language Generationcited by 347
  11. 2020Counterfactual Samples Synthesizing for Robust Visual Question Answeringcited by 153
  12. 2021Trankit: A Light-Weight Transformer-based Toolkit for Multilingual Natural Language Processingcited by 68
  13. 2020Consistency of a Recurrent Language Model With Respect to Incomplete Decodingcited by 54
  14. 2024InsightNet: Structured Insight Mining from Customer Feedbackcited by 4

Built on by

Not cited yet.