mlsanityData to 7 Oct 2026

Aligning MAGMA by Few-Shot Learning and Finetuning

arXiv 2210.14161 ↗First posted 18 Oct 2022cs.CV cs.AIcited by 1

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

also in 1 more
Abstract (arXiv)

The goal of vision-language modeling is to allow models to tie language understanding with visual inputs. The aim of this paper is to evaluate and align the Visual Language Model (VLM) called Multimodal Augmentation of Generative Models through Adapter-based finetuning (MAGMA) with human values. MAGMA is a VLM that is capable of image captioning and visual question-answering. We will evaluate its alignment in three different scenarios. To begin, we assess MAGMA's out-of-the-box alignment through the checkpoint provided by Hugging Face. Then, we measure if few-shot learning manages to improve the results. Finally, we finetune the model on aligned examples and evaluate its behavior.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 5 of its 8 references that are in the map. First posted on arXiv 18 Oct 2022.

Builds on

5 of 8 references.

References · 5

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,209
  3. 2020Aligning AI With Shared Human Valuescited by 706
  4. 2019Generalizing from a Few Examples: A Survey on Few-Shot Learningcited by 462
  5. 2021MAGMA -- Multimodal Augmentation of Generative Models through Adapter-based Finetuningcited by 85

Built on by

Cited by year

Citing papers per year: 2023: 1.

1 paper.

  1. 2023Towards ethical multimodal systemscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2023Contrastive Vision-Language Alignment Makes Efficient Instruction Learner
  2. 2025Feedback-Driven Vision-Language Alignment with Minimal Human Supervision
  3. 2023InfMLLM: A Unified Framework for Visual-Language Tasks
  4. 2024FiVL: A Framework for Improved Vision-Language Alignment through the Lens of Training, Evaluation and Explainability
  5. 2025Symmetrical Visual Contrastive Optimization: Aligning Vision-Language Models with Minimal Contrastive Images
  6. 2025Improved Alignment of Modalities in Large Vision Language Models
  7. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projection
  8. 2024CAST: Cross-modal Alignment Similarity Test for Vision Language Models