mlsanityData to 7 Oct 2026

A Foundational Multi-Modal Model for Few-Shot Learning

arXiv 2508.04746 ↗First posted 6 Aug 2025cs.LGcited by 0

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

Few-shot learning (FSL) is a machine learning paradigm that aims to generalize models from a small number of labeled examples, typically fewer than 10 per class. FSL is particularly crucial in biomedical, environmental, materials, and mechanical sciences, where samples are limited and data collection is often prohibitively costly, time-consuming, or ethically constrained. In this study, we present an innovative approach to FSL by demonstrating that a Large Multi-Modal Model (LMMM), trained on a set of independent tasks spanning diverse domains, task types, and input modalities, can substantially improve the generalization of FSL models, outperforming models based on conventional meta-learning on tasks of the same type. To support this, we first constructed a Multi-Modal Model Few-shot Dataset (M3FD, over 10K+ few-shot samples), which includes 2D RGB images, 2D/3D medical scans, tabular and time-course datasets, from which we manually curated FSL tasks such as classification. We further introduced M3F (Multi-Modal Model for Few-shot learning framework), a novel Large Multi-Modal Model framework tailored for data-constrained scientific applications. M3F supports a wide range of scientific data types through a modular pipeline. By fine-tuning the model on M3FD, M3F improves model performance, making LMMM feasible for real-world FSL deployment. The source code is located at https://github.com/ptdang1001/M3F. To democratize access to complex FSL data and promote reproducibility for public usage, M3FD is paired with a flexible and user-friendly tool that enables efficient querying, task-specific sampling, and preprocessing. Together, our dataset and framework offer a unified, scalable solution that significantly lowers the barrier to applying LMMMs in data-scarce scientific domains.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 17 of its 32 references that are in the map. First posted on arXiv 6 Aug 2025.

Builds on

17 of 32 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023A Survey on Multimodal Large Language Models23 Jun

Other references · 16

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  3. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  4. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  5. 2022Emergent Abilities of Large Language Modelscited by 2,483
  6. 2016Matching Networks for One Shot Learningcited by 2,246
  7. 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,427
  8. 2017Learning to Compare: Relation Network for Few-Shot Learningcited by 1,139
  9. 2022MaPLe: Multi-modal Prompt Learningcited by 685
  10. 2019Using Self-Supervised Learning Can Improve Model Robustness and Uncertaintycited by 511
  11. 2023Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Biascited by 230
  12. 2022A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunitiescited by 162
4 more
  1. 2024QuRating: Selecting High-Quality Data for Training Language Modelscited by 153
  2. 2023Increasing Diversity While Maintaining Accuracy: Text Data Generation with Large Language Models and Human Interventionscited by 121
  3. 2022Learning from Few Examples: A Summary of Approaches to Few-Shot Learningcited by 86
  4. 2025Physics Informed Constrained Learning of Dynamics from Static Datacited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Cross-Modal Mapping: Mitigating the Modality Gap for Few-Shot Image Classification
  2. 2024Few-shot Adaptation of Medical Vision-Language Models
  3. 2024Tiny models from tiny data: Textual and null-text inversion for few-shot distillation
  4. 2023Meta-DM: Applications of Diffusion Models on Few-Shot Learning
  5. 2024Making Large Vision Language Models to be Good Few-shot Learners
  6. 2026Decompose, Compare, and Decide: Multimodal LLMs are Implicit Few-Shot Learners
  7. 2024What Do You See? Enhancing Zero-Shot Image Classification with Multimodal Large Language Models
  8. 2025Solving Semi-Supervised Few-Shot Learning from an Auto-Annotation Perspective