# A Novel Trustworthy Video Summarization Algorithm Through a Mixture of LoRA Experts

arXiv: https://arxiv.org/abs/2503.06064 · HTML: https://mlsanity.com/paper/2503.06064/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Vision mixture-of-experts, one of 12 directions in Mixture-of-experts.
- Builds on 21 of its 33 references that are in the map.
- First posted on arXiv 8 Mar 2025.

## Placed in

- Mixture-of-experts / [Vision mixture-of-experts](https://mlsanity.com/moe/vision-mixture-of-experts/)

## Builds on (most cited first, 21 of 21)

- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding](https://mlsanity.com/paper/2306.02858/) (2023)
- [VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Tasks](https://mlsanity.com/paper/2305.11175/) (2023)
- [What matters when building vision-language models?](https://mlsanity.com/paper/2405.02246/) (2024)
- [VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks](https://mlsanity.com/paper/2410.05160/) (2024)
- [VideoLLM-online: Online Video Large Language Model for Streaming Video](https://mlsanity.com/paper/2406.11816/) (2024)
- [Video Summarization with Long Short-term Memory](https://mlsanity.com/paper/1605.08110/) (2016)
- [Mixture of Cluster-conditional LoRA Experts for Vision-language Instruction Tuning](https://mlsanity.com/paper/2312.12379/) (2023)
- [Low-Rank Few-Shot Adaptation of Vision-Language Models](https://mlsanity.com/paper/2405.18541/) (2024)
- [Video Summarization Using Deep Neural Networks: A Survey](https://mlsanity.com/paper/2101.06072/) (2021)
- [Align and Attend: Multimodal Summarization with Dual Contrastive Losses](https://mlsanity.com/paper/2303.07284/) (2023)
- [Video Transformers: A Survey](https://mlsanity.com/paper/2201.05991/) (2022)
- [VideoXum: Cross-modal Visual and Textural Summarization of Videos](https://mlsanity.com/paper/2303.12060/) (2023)
- [CondenseNet V2: Sparse Feature Reactivation for Deep Networks](https://mlsanity.com/paper/2104.04382/) (2021)
- [Benchmarking Robustness of Adaptation Methods on Pre-trained Vision-Language Models](https://mlsanity.com/paper/2306.02080/) (2023)
- [A Survey on Multi-modal Summarization](https://mlsanity.com/paper/2109.05199/) (2021)
- [GeoLLaVA: Efficient Fine-Tuned Vision-Language Models for Temporal Change Detection in Remote Sensing](https://mlsanity.com/paper/2410.19552/) (2024)
- [Abstractive Meeting Summarization: A Survey](https://mlsanity.com/paper/2208.04163/) (2022)
- [Multi-Stream Dynamic Video Summarization](https://mlsanity.com/paper/1812.00108/) (2018)
- [OStr-DARTS: Differentiable Neural Architecture Search based on Operation Strength](https://mlsanity.com/paper/2409.14433/) (2024)

## Abstract (arXiv)

With the exponential growth of user-generated content on video-sharing platforms, the challenge of facilitating efficient searching and browsing of videos has garnered significant attention. To enhance users' ability to swiftly locate and review pertinent videos, the creation of concise and informative video summaries has become increasingly important. Video-llama is an effective tool for generating video summarization, but it cannot effectively unify and optimize the modeling of temporal and spatial features and requires a lot of computational resources and time. Therefore, we propose MiLoRA-ViSum to more efficiently capture complex temporal dynamics and spatial relationships inherent in video data and to control the number of parameters for training. By extending traditional Low-Rank Adaptation (LoRA) into a sophisticated mixture-of-experts paradigm, MiLoRA-ViSum incorporates a dual temporal-spatial adaptation mechanism tailored specifically for video summarization tasks. This approach dynamically integrates specialized LoRA experts, each fine-tuned to address distinct temporal or spatial dimensions. Extensive evaluations of the VideoXum and ActivityNet datasets demonstrate that MiLoRA-ViSum achieves the best summarization performance compared to state-of-the-art models, while maintaining significantly lower computational costs. The proposed mixture-of-experts strategy, combined with the dual adaptation mechanism, highlights the model's potential to enhance video summarization capabilities, particularly in large-scale applications requiring both efficiency and precision.