mlsanityData to 7 Oct 2026

Multi-Agent Cooperative Learning for Robust Vision-Language Alignment under OOD Concepts

arXiv 2601.09746 ↗First posted 11 Jan 2026cs.MAcited by 0

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

This paper introduces a novel Multi-Agent Cooperative Learning (MACL) framework to address cross-modal alignment collapse in vision-language models when handling out-of-distribution (OOD) concepts. Four core agents, including image, text, name, and coordination agents, collaboratively mitigate modality imbalance through structured message passing. The proposed framework enables multi-agent feature space name learning, incorporates a context exchange enhanced few-shot learning algorithm, and adopts an adaptive dynamic balancing mechanism to regulate inter-agent contributions. Experiments on the VISTA-Beyond dataset demonstrate that MACL significantly improves performance in both few-shot and zero-shot settings, achieving 1-5% precision gains across diverse visual domains.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 41 of its 60 references that are in the map. First posted on arXiv 11 Jan 2026.

Builds on

41 of 60 references.

References · 41

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2023Visual Instruction Tuningcited by 7,792
  5. 2024Qwen2.5 Technical Reportcited by 4,713
  6. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  7. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  8. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  9. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  10. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  11. 2022Visual Prompt Tuningcited by 1,674
  12. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
29 more
  1. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
  2. 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,427
  3. 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapterscited by 1,021
  4. 2021Robust fine-tuning of zero-shot modelscited by 785
  5. 2022A Survey on In-context Learningcited by 717
  6. 2021Towards Out-Of-Distribution Generalization: A Surveycited by 431
  7. 2021How Much Can CLIP Benefit Vision-and-Language Tasks?cited by 321
  8. 2021AdaViT: Adaptive Vision Transformers for Efficient Image Recognitioncited by 228
  9. 2024Unveiling Encoder-Free Vision-Language Modelscited by 90
  10. 2024Boosting Vision-Language Models with Transductioncited by 36
  11. 2023Distilling Large Vision-Language Model with Out-of-Distribution Generalizabilitycited by 33
  12. 2025DrDiff: Dynamic Routing Diffusion with Hierarchical Attention for Breaking the Efficiency-Quality Trade-offcited by 33
  13. 2025MM-CoT:A Benchmark for Probing Visual Chain-of-Thought Reasoning in Multimodal Modelscited by 33
  14. 2023DepthSSC: Monocular 3D Semantic Scene Completion via Depth-Spatial Alignment and Voxel Adaptationcited by 30
  15. 2025OSC: Cognitive Orchestration through Dynamic Knowledge Alignment in Multi-Agent LLM Collaborationcited by 29
  16. 2024Out-of-distribution generalization via composition: a lens through induction heads in Transformerscited by 28
  17. 2024Cascade Prompt Learning for Vision-Language Model Adaptationcited by 19
  18. 2025Failure-Driven Workflow Refinementcited by 19
  19. 2025Hybrid Token Compression for Vision-Language Modelscited by 19
  20. 2025Kolmogorov-Arnold Fourier Networkscited by 18
  21. 2025Learning Dynamics of VLM Finetuningcited by 17
  22. 2025Top-Down Semantic Refinement for Image Captioningcited by 16
  23. 2025LLM-CAS: Dynamic Neuron Perturbation for Real-Time Hallucination Correctioncited by 15
  24. 2023Learning to Name Classes for Vision and Language Modelscited by 12
  25. 2023Exploring Large Language Models for Multi-Modal Out-of-Distribution Detectioncited by 12
  26. 2024LLaVA-Read: Enhancing Reading Ability of Multimodal Language Modelscited by 12
  27. 2023Robust Fine-Tuning of Vision-Language Models for Domain Generalizationcited by 7
  28. 2017Context encoders as a simple but powerful extension of word2veccited by 3
  29. 2024MTA: Multimodal Task Alignment for BEV Perception and Captioningcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Bridging the Semantic Chasm: Synergistic Conceptual Anchoring for Generalized Few-Shot and Zero-Shot OOD Perception
  2. 2026Scaling Video Understanding via Compact Latent Multi-Agent Collaboration
  3. 2026MAIL++: Multi-Modal Bi-directional Agent Layer for Vision-Language Models
  4. 2022Vision-Language Intelligence: Tasks, Representation Learning, and Large Models
  5. 2026Seeing Before Agreeing: Aligning Multi-Agent Consensus with Visual Evidence
  6. 2025VL-SAE: Interpreting and Enhancing Vision-Language Alignment with a Unified Concept Set
  7. 2022MVPTR: Multi-Level Semantic Alignment for Vision-Language Pre-Training via Multi-Stage Learning
  8. 2026Cooperative Multi-Agent Vision-Language-Action Models via Reinforced Fine Tuning