Web-Scale Multimodal Summarization using CLIP-Based Semantic Alignment
Not on the map. Why?
Abstract (arXiv)
We introduce Web-Scale Multimodal Summarization, a lightweight framework for generating summaries by combining retrieved text and image data from web sources. Given a user-defined topic, the system performs parallel web, news, and image searches. Retrieved images are ranked using a fine-tuned CLIP model to measure semantic alignment with topic and text. Optional BLIP captioning enables image-only summaries for stronger multimodal coherence.The pipeline supports features such as adjustable fetch limits, semantic filtering, summary styling, and downloading structured outputs. We expose the system via a Gradio-based API with controllable parameters and preconfigured presets.Evaluation on 500 image-caption pairs with 20:1 contrastive negatives yields a ROC-AUC of 0.9270, an F1-score of 0.6504, and an accuracy of 96.99%, demonstrating strong multimodal alignment. This work provides a configurable, deployable tool for web-scale summarization that integrates language, retrieval, and vision models in a user-extensible pipeline.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 7 of its 17 references that are in the map. First posted on arXiv 16 Feb 2026.
Builds on
7 of 17 references.
References · 7
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2023Visual Instruction Tuningcited by 7,869
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
- 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
- 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,102
- 2019Text Summarization with Pretrained Encoderscited by 556
- 2020VMSMO: Learning to Generate Multimodal Summary for Video-based News Articlescited by 44
Built on by
Not cited yet.