mlsanityData to 14 Oct 2026

SAMA: Semantic Anchor-aligned Augmentation for Unified Low-Resource Multimodal Information Extraction

arXiv 2606.18780 ↗First posted 17 Jun 2026cs.CV cs.CL cs.MMcited by 0

Not on the map. Why?

Abstract (arXiv)

Multimodal Information Extraction (MIE)-covering tasks such as Multimodal Named Entity Recognition (MNER), Relation Extraction (MRE), and Event Extraction (MEE)-is essential for understanding multimedia content but remains constrained by severe data scarcity. Although data augmentation is a promising remedy, existing approaches are impeded by coarse cross-modal alignment and fragmented, task-specific designs that fail to exploit shared semantic knowledge. To overcome these limitations, we introduce Semantic Anchor-aligned Multimodal Augmentation (SAMA), a unified framework for generating high-fidelity, task-aware synthetic data. SAMA constructs structured semantic anchors from ground-truth labels to guide a Collaborative Multi-Experts Multimodal Large Language Model (CME-MLLM), which integrates a Universal Adapter for shared semantics with Task-Specific Adapters to produce diverse yet constraint-compliant textual samples. For image synthesis, SAMA employs an Anchor-Preserving Diffusion mechanism that uses anchor-weighted prompts and latent conditioning to maintain critical semantic anchors while diversifying visual contexts. To eliminate the need for manual verification, SAMA further introduces a Dual-Constraint Filtering module that selects synthetic samples based on both cross-modal consistency and anchor fidelity. Extensive experiments across benchmark datasets for MNER, MRE, and MEE demonstrate that SAMA consistently outperforms state-of-the-art augmentation baselines under both fully supervised and low-resource settings, underscoring its versatility, robustness, and effectiveness.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 30 of its 61 references that are in the map. First posted on arXiv 17 Jun 2026.

Builds on

30 of 61 references.

References · 30

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  5. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  6. 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 807
  7. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 691
  8. 2023Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflictscited by 341
  9. 2022CLIP-Event: Connecting Text and Images with Event Structurescited by 86
  10. 2020An Analysis of Simple Data Augmentation for Named Entity Recognitioncited by 76
  11. 2023Automatic Evaluation of Attribution by Large Language Modelscited by 72
  12. 2020Cross-media Structured Common Space for Multimedia Event Extractioncited by 61
18 more
  1. 2022MixGen: A New Multi-Modal Data Augmentationcited by 61
  2. 2019Data Augmentation via Dependency Tree Morphing for Low-Resource Languagescited by 45
  3. 2022Hybrid Transformer with Multi-level Fusion for Multimodal Knowledge Graph Completioncited by 44
  4. 2018Multimodal Named Entity Recognition for Short Social Media Postscited by 43
  5. 2022Named Entity and Relation Extraction with Multi-Modal Retrievalcited by 26
  6. 2021ITA: Image-Text Alignments for Multi-Modal Named Entity Recognitioncited by 24
  7. 2024UMIE: Unified Multimodal Information Extraction with Instruction Tuningcited by 17
  8. 2023Prompting ChatGPT in MNER: Enhanced Multimodal Named Entity Recognition with Auxiliary Refined Knowledgecited by 11
  9. 2023Training Multimedia Event Extraction With Generated Images and Captionscited by 11
  10. 2023Document-level Relation Extraction with Cross-sentence Reasoning Graphcited by 9
  11. 2023Learning Implicit Entity-object Relations by Bidirectional Generative Alignment for Multimodal NERcited by 8
  12. 2023Multimodal Relation Extraction with Cross-Modal Retrieval and Synthesiscited by 7
  13. 2025Collaborative Multi-LoRA Experts with Achievement-based Multi-Tasks Loss for Unified Multimodal Information Extractioncited by 7
  14. 2023Chain-of-Thought Prompt Distillation for Multimodal Named Entity Recognition and Multimodal Relation Extractioncited by 6
  15. 2024FCDS: Fusing Constituency and Dependency Syntax into Document-Level Relation Extractioncited by 5
  16. 2024BANER: Boundary-Aware LLMs for Few-Shot Named Entity Recognitioncited by 3
  17. 2025Bridging Generative and Discriminative Learning: Few-Shot Relation Extraction via Two-Stage Knowledge-Guided Pre-trainingcited by 3
  18. 2025Extracting Events Like Code: A Multi-Agent Programming Framework for Zero-Shot Event Extractioncited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025REMOTE: A Unified Multimodal Relation Extraction Framework with Multilevel Optimal Transport and Mixture-of-Experts
  2. 2024Semantic Alignment for Multimodal Large Language Models
  3. 2025MANTA: Cross-Modal Semantic Alignment and Information-Theoretic Optimization for Long-form Multimodal Understanding
  4. 2026Lens: Bringing the Right Semantic Perspective into Focus for Training-Free Multimodal Representation Learning
  5. 2024Recognizing Everything from All Modalities at Once: Grounded Multimodal Universal Information Extraction
  6. 2025Knowledge Completes the Vision: A Multimodal Entity-aware Retrieval-Augmented Generation Framework for News Image Captioning
  7. 2024UniMEL: A Unified Framework for Multimodal Entity Linking with Large Language Models
  8. 2025DeepMEL: A Multi-Agent Collaboration Framework for Multimodal Entity Linking