# SAMA: Semantic Anchor-aligned Augmentation for Unified Low-Resource Multimodal Information Extraction

arXiv: https://arxiv.org/abs/2606.18780 · HTML: https://mlsanity.com/paper/2606.18780/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 61 references that are in the map.
- First posted on arXiv 17 Jun 2026.

## Builds on (most cited first, 30 of 30)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning](https://mlsanity.com/paper/2305.06500/) (2023)
- [EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks](https://mlsanity.com/paper/1901.11196/) (2019)
- [EVA: Exploring the Limits of Masked Visual Representation Learning at Scale](https://mlsanity.com/paper/2211.07636/) (2022)
- [Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts](https://mlsanity.com/paper/2305.13300/) (2023)
- [CLIP-Event: Connecting Text and Images with Event Structures](https://mlsanity.com/paper/2201.05078/) (2022)
- [An Analysis of Simple Data Augmentation for Named Entity Recognition](https://mlsanity.com/paper/2010.11683/) (2020)
- [Automatic Evaluation of Attribution by Large Language Models](https://mlsanity.com/paper/2305.06311/) (2023)
- [Cross-media Structured Common Space for Multimedia Event Extraction](https://mlsanity.com/paper/2005.02472/) (2020)
- [MixGen: A New Multi-Modal Data Augmentation](https://mlsanity.com/paper/2206.08358/) (2022)
- [Data Augmentation via Dependency Tree Morphing for Low-Resource Languages](https://mlsanity.com/paper/1903.09460/) (2019)
- [Hybrid Transformer with Multi-level Fusion for Multimodal Knowledge Graph Completion](https://mlsanity.com/paper/2205.02357/) (2022)
- [Multimodal Named Entity Recognition for Short Social Media Posts](https://mlsanity.com/paper/1802.07862/) (2018)
- [Named Entity and Relation Extraction with Multi-Modal Retrieval](https://mlsanity.com/paper/2212.01612/) (2022)
- [ITA: Image-Text Alignments for Multi-Modal Named Entity Recognition](https://mlsanity.com/paper/2112.06482/) (2021)
- [UMIE: Unified Multimodal Information Extraction with Instruction Tuning](https://mlsanity.com/paper/2401.03082/) (2024)
- [Prompting ChatGPT in MNER: Enhanced Multimodal Named Entity Recognition with Auxiliary Refined Knowledge](https://mlsanity.com/paper/2305.12212/) (2023)
- [Training Multimedia Event Extraction With Generated Images and Captions](https://mlsanity.com/paper/2306.08966/) (2023)
- [Document-level Relation Extraction with Cross-sentence Reasoning Graph](https://mlsanity.com/paper/2303.03912/) (2023)
- [Learning Implicit Entity-object Relations by Bidirectional Generative Alignment for Multimodal NER](https://mlsanity.com/paper/2308.02570/) (2023)
- [Multimodal Relation Extraction with Cross-Modal Retrieval and Synthesis](https://mlsanity.com/paper/2305.16166/) (2023)
- [Collaborative Multi-LoRA Experts with Achievement-based Multi-Tasks Loss for Unified Multimodal Information Extraction](https://mlsanity.com/paper/2505.06303/) (2025)
- [Chain-of-Thought Prompt Distillation for Multimodal Named Entity Recognition and Multimodal Relation Extraction](https://mlsanity.com/paper/2306.14122/) (2023)
- [FCDS: Fusing Constituency and Dependency Syntax into Document-Level Relation Extraction](https://mlsanity.com/paper/2403.01886/) (2024)
- [BANER: Boundary-Aware LLMs for Few-Shot Named Entity Recognition](https://mlsanity.com/paper/2412.02228/) (2024)
- [Bridging Generative and Discriminative Learning: Few-Shot Relation Extraction via Two-Stage Knowledge-Guided Pre-training](https://mlsanity.com/paper/2505.12236/) (2025)
- [Extracting Events Like Code: A Multi-Agent Programming Framework for Zero-Shot Event Extraction](https://mlsanity.com/paper/2511.13118/) (2025)

## Abstract (arXiv)

Multimodal Information Extraction (MIE)-covering tasks such as Multimodal Named Entity Recognition (MNER), Relation Extraction (MRE), and Event Extraction (MEE)-is essential for understanding multimedia content but remains constrained by severe data scarcity. Although data augmentation is a promising remedy, existing approaches are impeded by coarse cross-modal alignment and fragmented, task-specific designs that fail to exploit shared semantic knowledge. To overcome these limitations, we introduce Semantic Anchor-aligned Multimodal Augmentation (SAMA), a unified framework for generating high-fidelity, task-aware synthetic data. SAMA constructs structured semantic anchors from ground-truth labels to guide a Collaborative Multi-Experts Multimodal Large Language Model (CME-MLLM), which integrates a Universal Adapter for shared semantics with Task-Specific Adapters to produce diverse yet constraint-compliant textual samples. For image synthesis, SAMA employs an Anchor-Preserving Diffusion mechanism that uses anchor-weighted prompts and latent conditioning to maintain critical semantic anchors while diversifying visual contexts. To eliminate the need for manual verification, SAMA further introduces a Dual-Constraint Filtering module that selects synthetic samples based on both cross-modal consistency and anchor fidelity. Extensive experiments across benchmark datasets for MNER, MRE, and MEE demonstrate that SAMA consistently outperforms state-of-the-art augmentation baselines under both fully supervised and low-resource settings, underscoring its versatility, robustness, and effectiveness.