mlsanityData to 14 Oct 2026

MAG: MAnifold Guided Semi-Supervised Multi-modal In-Context Learning

arXiv 2608.12724 ↗First posted 13 Aug 2026cs.LGcited by 0

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

Few-shot in-context learning (ICL) with multi-modal large language models (MLLMs) enables task adaptation without parameter updates, but its performance is highly sensitive to the quality and coverage of the selected demonstrations. While unlabeled multi-modal data is abundant, it remains elusive how to exploit them for ICL. We propose MAG (MAnifold-Guided semi-supervised in-context demonstra- tion selection), an efficient framework that leverages unlabeled data to improve multi-modal ICL. MAG formulates demonstration selection as a semi-supervised propagation problem on a multi-modal graph and adopts a two-stage strategy: (i) relevance score propagation identifies a compact set of high-impact unlabeled samples for pseudo-labeling, reducing MLLM inference cost; (ii) multi-modal relevance is used to select the final demonstrations. We show that textual represen- tations are more effective for relevance propagation, while both visual and textual modalities are crucial for high-quality demonstration selection. Experiments on eight multi-modal benchmarks demonstrate that MAG consistently outperforms strong baselines in label-scarce regimes, achieving significant gains with a limited pseudo-labeling budget.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 40 of its 49 references that are in the map. First posted on arXiv 13 Aug 2026.

Builds on

40 of 49 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025TACO: Enhancing Multimodal In-context Learning via Task Mapping-Guided Sequence Configuration21 May
  2. 2023Can Multimodal Large Language Models Truly Perform Multimodal In-Context Learning?29 Nov

Other references · 38

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2023Visual Instruction Tuningcited by 7,869
  5. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  6. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  7. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  8. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,085
  9. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,216
  10. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,135
  11. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,121
  12. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,045
26 more
  1. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 938
  2. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 914
  3. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 896
  4. 2022What Can Transformers Learn In-Context? A Case Study of Simple Function Classescited by 715
  5. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 653
  6. 2021Learning To Retrieve Prompts for In-Context Learningcited by 648
  7. 2019Label Propagation for Deep Semi-supervised Learningcited by 289
  8. 2022Self-Adaptive In-Context Learning: An Information Compression Perspective for In-Context Example Selection and Orderingcited by 161
  9. 2023Compositional Exemplars for In-context Learningcited by 144
  10. 2022Diverse Demonstrations Improve In-context Compositional Generalizationcited by 139
  11. 2024Visual In-Context Learning for Large Vision-Language Modelscited by 128
  12. 2022Learning to Learn with Generative Models of Neural Network Checkpointscited by 86
  13. 2023EmoSet: A Large-scale Visual Emotion Dataset with Rich Attributescited by 82
  14. 2024Enhancing Large Vision Language Models with Self-Training on Image Comprehensioncited by 70
  15. 2023Large Language Models are Visual Reasoning Coordinatorscited by 58
  16. 2023How Do In-Context Examples Affect Compositional Generalization?cited by 51
  17. 2016Efficient Diffusion on Region Manifolds: Recovering Small Objects with Compact CNN Representationscited by 50
  18. 2018Contemplating Visual Emotions: Understanding and Overcoming Dataset Biascited by 41
  19. 2024VL-ICL Bench: The Devil in the Details of Multimodal In-Context Learningcited by 36
  20. 2023How to Configure Good In-Context Sequence for Visual Question Answeringcited by 33
  21. 2024Enhancing Advanced Visual Reasoning Ability of Large Language Modelscited by 18
  22. 2025MAPLE: Many-Shot Adaptive Pseudo-Labeling for In-Context Learningcited by 13
  23. 2023SimMatchV2: Semi-Supervised Learning with Graph Consistencycited by 12
  24. 2025Provoking Multi-modal Few-Shot LVLM via Exploration-Exploitation In-Context Learningcited by 7
  25. 2025MPCAR: Multi-Perspective Contextual Augmentation for Enhanced Visual Reasoning in Large Vision-Language Modelscited by 1
  26. 2025From Cross-Task Examples to In-Task Prompts: A Graph-Based Pseudo-Labeling Framework for In-context Learningcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024AIM: Let Any Multi-modal Large Language Models Embrace Efficient In-Context Learning
  2. 2023MMICT: Boosting Multi-Modal Fine-Tuning with In-Context Examples
  3. 2024From Introspection to Best Practices: Principled Analysis of Demonstrations in Multimodal In-Context Learning
  4. 2024What Factors Affect Multi-Modal In-Context Learning? An In-Depth Exploration
  5. 2026Why Multimodal In-Context Learning Lags Behind? Unveiling the Inner Mechanisms and Bottlenecks
  6. 2023Link-Context Learning for Multimodal LLMs
  7. 2025Unveiling Effective In-Context Configurations for Image Captioning: An External & Internal Analysis
  8. 2024Does In-Context Learning Really Learn? Rethinking How Large Language Models Respond and Solve Tasks via In-Context Learning