mlsanityData to 14 Oct 2026

Cross-media Structured Common Space for Multimedia Event Extraction

arXiv 2005.02472 ↗First posted 5 May 2020cs.MM cs.CL cs.CVcited by 61

Not on the map. Why?

Abstract (arXiv)

We introduce a new task, MultiMedia Event Extraction (M2E2), which aims to extract events and their arguments from multimedia documents. We develop the first benchmark and collect a dataset of 245 multimedia news articles with extensively annotated events and arguments. We propose a novel method, Weakly Aligned Structured Embedding (WASE), that encodes structured representations of semantic information from textual and visual data into a common embedding space. The structures are aligned across modalities by employing a weakly supervised training strategy, which enables exploiting available resources without explicit cross-media annotation. Compared to uni-modal state-of-the-art methods, our approach achieves 4.0% and 9.8% absolute F-score gains on text event argument role labeling and visual event extraction. Compared to state-of-the-art multimedia unstructured representations, we achieve 8.3% and 5.0% absolute F-score gains on multimedia event extraction and argument role labeling, respectively. By utilizing images, we extract 21.4% more event mentions than traditional text-only methods.

Key facts. Cited by 61 papers in the map (data through 14 Oct 2026). First posted on arXiv 5 May 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 8; 2022: 10; 2023: 14; 2024: 11; 2025: 9; 2026: 9.

61 papers.

Sort:

By area:

  1. 2022Translation between Molecules and Natural Languagecited by 177
  2. 2022CLIP-Event: Connecting Text and Images with Event Structurescited by 86
  3. 2022Multi-Modal Knowledge Graph Construction and Application: A Surveycited by 74
  4. 2021Visual Semantic Role Labeling for Video Understandingcited by 60
  5. 2023Multimodal Automated Fact-Checking: A Surveycited by 53
  6. 2023Do LVLMs Understand Charts? Analyzing and Correcting Factual Errors in Chart Captioningcited by 49
  7. 2023Language Models Can Improve Event Prediction by Few-Shot Abductive Reasoningcited by 45
  8. 2023TextEE: Benchmark, Reevaluation, Reflections, and Future Challenges in Event Extractioncited by 28
  9. 2021Joint Multimedia Event Extraction from Video and Articlecited by 19
  10. 2021MuMuQA: Multimedia Multi-Hop News Question Answering via Cross-Media Knowledge Extraction and Groundingcited by 19
  11. 2021What is Event Knowledge Graph: A Surveycited by 19
  12. 2021EventPlus: A Temporal Event Understanding Pipelinecited by 17
38 more
  1. 2024UMIE: Unified Multimodal Information Extraction with Instruction Tuningcited by 17
  2. 2021Video Question Answering with Phrases via Semantic Rolescited by 13
  3. 2023ViStruct: Visual Structural Knowledge Extraction via Curriculum Guided Code-Vision Representationcited by 13
  4. 2021A Survey on Deep Learning Event Extraction: Approaches and Applicationscited by 12
  5. 2024A Survey on Integrated Sensing, Communication, and Computationcited by 12
  6. 2023Training Multimedia Event Extraction With Generated Images and Captionscited by 11
  7. 2022Information Extraction in Low-Resource Scenarios: Survey and Perspectivecited by 10
  8. 2022Video Event Extraction via Tracking Visual States of Argumentscited by 8
  9. 2023MultiVENT: Multilingual Videos of Events with Aligned Natural Textcited by 7
  10. 2024Improving Event Definition Following For Zero-Shot Event Detectioncited by 7
  11. 2024Event Detection from Social Media for Epidemic Predictioncited by 7
  12. 2024SPEED++: A Multilingual Event Extraction Framework for Epidemic Prediction and Preparednesscited by 7
  13. 2025Collaborative Multi-LoRA Experts with Achievement-based Multi-Tasks Loss for Unified Multimodal Information Extractioncited by 7
  14. 2023Understanding Social Media Cross-Modality Discourse in Linguistic Spacecited by 6
  15. 2023Multimodal Question Answering for Unified Information Extractioncited by 6
  16. 2024MMUTF: Multimodal Multimedia Event Argument Extraction with Unified Template Fillingcited by 6
  17. 2025Event Extraction in Large Language Modelcited by 5
  18. 2022Fine-Grained Visual Entailmentcited by 4
  19. 2024Multimodal Cross-Document Event Coreference Resolution Using Linear Semantic Transfer and Mixed-Modality Ensemblescited by 4
  20. 2021Fine-Grained Chemical Entity Typing with Multimodal Knowledge Representationcited by 3
  21. 2022Beyond Grounding: Extracting Fine-Grained Event Hierarchies Across Modalitiescited by 3
  22. 2022Vision and Structured-Language Pretraining for Cross-Modal Food Retrievalcited by 3
  23. 2025SNaRe: Domain-aware Data Generation for Low-Resource Event Detectioncited by 3
  24. 2025SEOE: A Scalable and Reliable Semantic Evaluation Framework for Open Domain Event Detectioncited by 3
  25. 2025Benchmarking and Improving LVLMs on Event Extraction from Multimedia Documentscited by 3
  26. 2022Detecting the Role of an Entity in Harmful Memes: Techniques and Their Limitationscited by 2
  27. 2022Language Model Pre-Training with Sparse Latent Typingcited by 2
  28. 2024GenEARL: A Training-Free Generative Framework for Multimodal Event Argument Role Labelingcited by 2
  29. 2024ARMADA: Attribute-Based Multimodal Data Augmentationcited by 2
  30. 2023Few-shot Domain-Adaptive Visually-fused Event Detection from Textcited by 1
  31. 2023Artificial Intelligence for Emergency Responsecited by 1
  32. 2023STELLA: Continual Audio-Video Pre-training with Spatio-Temporal Localized Alignmentcited by 1
  33. 2023RESIN-EDITOR: A Schema-guided Hierarchical Event Graph Visualizer and Editorcited by 1
  34. 2024EXCEEDS: Extracting Complex Events via Nugget-based Grid Modeling in Scientific Domaincited by 1
  35. 2025Retrieval-Enhanced Few-Shot Prompting for Speech Event Extractioncited by 1
  36. 2025Multimodal Event Detection: Current Approaches and Defining the New Playground through LLMs and VLMscited by 1
  37. 2023In Defense of Structural Symbolic Representation for Video Event-Relation Predictioncited by 0
  38. 2024Beyond Exact Match: Semantically Reassessing Event Extraction by Large Language Modelscited by 0

50 most cited of 61 All 61 (JSON)