# Benchmarking and Improving LVLMs on Event Extraction from Multimedia Documents

arXiv: https://arxiv.org/abs/2509.12876 · HTML: https://mlsanity.com/paper/2509.12876/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 37 references that are in the map.
- First posted on arXiv 16 Sep 2025.

## Builds on (most cited first, 15 of 15)

- [DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding](https://mlsanity.com/paper/2412.10302/) (2024)
- [AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actions](https://mlsanity.com/paper/1705.08421/) (2017)
- [Long-Term Feature Banks for Detailed Video Understanding](https://mlsanity.com/paper/1812.05038/) (2018)
- [CLIP-Event: Connecting Text and Images with Event Structures](https://mlsanity.com/paper/2201.05078/) (2022)
- [Grounded Situation Recognition](https://mlsanity.com/paper/2003.12058/) (2020)
- [Jointly Multiple Events Extraction via Attention-based Graph Information Aggregation](https://mlsanity.com/paper/1809.09078/) (2018)
- [Cross-media Structured Common Space for Multimedia Event Extraction](https://mlsanity.com/paper/2005.02472/) (2020)
- [Using Clinical Notes with Time Series Data for ICU Management](https://mlsanity.com/paper/1909.09702/) (2019)
- [ADELIE: Aligning Large Language Models on Information Extraction](https://mlsanity.com/paper/2405.05008/) (2024)
- [Collaborative Transformers for Grounded Situation Recognition](https://mlsanity.com/paper/2203.16518/) (2022)
- [UMIE: Unified Multimodal Information Extraction with Instruction Tuning](https://mlsanity.com/paper/2401.03082/) (2024)
- [Training Multimedia Event Extraction With Generated Images and Captions](https://mlsanity.com/paper/2306.08966/) (2023)
- [OmniEvent: A Comprehensive, Fair, and Easy-to-Use Toolkit for Event Understanding](https://mlsanity.com/paper/2309.14258/) (2023)
- [EventNet: A Large Scale Structured Concept Library for Complex Event Detection in Video](https://mlsanity.com/paper/1506.02328/) (2015)
- [Event Extraction with Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1804.07881/) (2018)

## Built on by (most cited first, 3 of 3)

- [RMPL: Relation-aware Multi-task Progressive Learning with Stage-wise Training for Multimedia Event Extraction](https://mlsanity.com/paper/2602.13748/) (2026)
- [NEST: Narrative Event Structures in Time for Long Video Understanding](https://mlsanity.com/paper/2606.19706/) (2026)
- [Evaluation Pitfalls and Challenges in Multimedia Event Extraction](https://mlsanity.com/paper/2606.26775/) (2026)

## Abstract (arXiv)

The proliferation of multimedia content necessitates the development of effective Multimedia Event Extraction (M2E2) systems. Though Large Vision-Language Models (LVLMs) have shown strong cross-modal capabilities, their utility in the M2E2 task remains underexplored. In this paper, we present the first systematic evaluation of representative LVLMs, including DeepSeek-VL2 and the Qwen-VL series, on the M2E2 dataset. Our evaluations cover text-only, image-only, and cross-media subtasks, assessed under both few-shot prompting and fine-tuning settings. Our key findings highlight the following valuable insights: (1) Few-shot LVLMs perform notably better on visual tasks but struggle significantly with textual tasks; (2) Fine-tuning LVLMs with LoRA substantially enhances model performance; and (3) LVLMs exhibit strong synergy when combining modalities, achieving superior performance in cross-modal settings. We further provide a detailed error analysis to reveal persistent challenges in areas such as semantic precision, localization, and cross-modal grounding, which remain critical obstacles for advancing M2E2 capabilities.