mlsanityData to 14 Oct 2026

Web-Scale Multimodal Summarization using CLIP-Based Semantic Alignment

arXiv 2602.14889 ↗First posted 16 Feb 2026cs.LG cs.CV cs.ETcited by 0

Not on the map. Why?

Abstract (arXiv)

We introduce Web-Scale Multimodal Summarization, a lightweight framework for generating summaries by combining retrieved text and image data from web sources. Given a user-defined topic, the system performs parallel web, news, and image searches. Retrieved images are ranked using a fine-tuned CLIP model to measure semantic alignment with topic and text. Optional BLIP captioning enables image-only summaries for stronger multimodal coherence.The pipeline supports features such as adjustable fetch limits, semantic filtering, summary styling, and downloading structured outputs. We expose the system via a Gradio-based API with controllable parameters and preconfigured presets.Evaluation on 500 image-caption pairs with 20:1 contrastive negatives yields a ROC-AUC of 0.9270, an F1-score of 0.6504, and an accuracy of 96.99%, demonstrating strong multimodal alignment. This work provides a configurable, deployable tool for web-scale summarization that integrates language, retrieval, and vision models in a user-extensible pipeline.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 7 of its 17 references that are in the map. First posted on arXiv 16 Feb 2026.

Builds on

7 of 17 references.

References · 7

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023Visual Instruction Tuningcited by 7,869
  3. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  4. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  5. 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,102
  6. 2019Text Summarization with Pretrained Encoderscited by 556
  7. 2020VMSMO: Learning to Generate Multimodal Summary for Video-based News Articlescited by 44

Built on by

Not cited yet.