mlsanityData to 14 Oct 2026

UniICL: Systematizing Unified Multimodal In-context Learning through a Capability-Oriented Taxonomy

arXiv 2603.24690 ↗First posted 25 Mar 2026cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

In-context learning (ICL) enables fast task adaptation from demonstrations without per-task parameter updates but remains highly sensitive to example selection and formatting. In unified multimodal models spanning understanding and generation, this sensitivity is exacerbated by cross-modal interference and varying cognitive demands. Consequently, in-context learning efficacy is often non-monotonic and highly task-dependent. To diagnose these behaviors, we introduce a six-level Capability-Oriented Taxonomy that categorizes the functional role of demonstrations from basic perception to high-order discernment. Guided by this cognitive framework, we construct UniICL-760K, a large-scale corpus featuring curated 8-shot in-context learning episodes across 15 subtasks, alongside UniICL-Bench for rigorous, controlled evaluation. We show that this data-driven assembly is the primary source of our gains. As a complementary, lightweight stabilizer, we additionally propose the Context-Adaptive Prototype Modulator, a plug-and-play module that further improves few-shot stability. Evaluations on UniICL-Bench show that our approach yields highly competitive unified results, outperforming larger-parameter multimodal large language model baselines on most understanding in-context learning tasks. Data and code are available at https://github.com/xuyicheng-zju/UniICL.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 55 of its 66 references that are in the map. First posted on arXiv 25 Mar 2026.

Builds on

55 of 66 references.

References · 55

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  3. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  4. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  5. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  6. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  7. 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
  8. 2025Qwen3-VL Technical Reportcited by 2,793
  9. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  10. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  11. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  12. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
38 more
  1. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
  2. 2025DINOv3cited by 1,414
  3. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
  4. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,261
  5. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,135
  6. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 972
  7. 2022In-context Learning and Induction Headscited by 964
  8. 2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehensioncited by 940
  9. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 938
  10. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
  11. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 897
  12. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 868
  13. 2024Emu3: Next-Token Prediction is All You Needcited by 781
  14. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 739
  15. 2022Transformers learn in-context by gradient descentcited by 732
  16. 2023VILA: On Pre-training for Visual Language Modelscited by 692
  17. 2023OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Modelscited by 565
  18. 2023Otter: A Multi-Modal Model with In-Context Instruction Tuningcited by 550
  19. 2023Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levelscited by 503
  20. 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learningcited by 466
  21. 2023Generative Multimodal Models are In-Context Learnerscited by 446
  22. 2024What matters when building vision-language models?cited by 326
  23. 2023Recognize Anything: A Strong Image Tagging Modelcited by 309
  24. 2023DreamLLM: Synergistic Multimodal Comprehension and Creationcited by 297
  25. 2025Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Freecited by 283
  26. 2023MIMIC-IT: Multi-Modal In-Context Instruction Tuningcited by 267
  27. 2023Generating Images with Multimodal Language Modelscited by 260
  28. 2025HPSv3: Towards Wide-Spectrum Human Preference Scorecited by 211
  29. 2023MMICL: Empowering Vision-language Model with Multi-Modal In-Context Learningcited by 172
  30. 2024GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generationcited by 153
  31. 2023Planting a SEED of Vision in Large Language Modelcited by 140
  32. 2025Detect Anything via Next Point Predictioncited by 73
  33. 2025Ovis-U1 Technical Reportcited by 67
  34. 2025UniWorld-V1: High-Resolution Semantic Encoders for Unified Visual Understanding and Generationcited by 66
  35. 2023CoDi-2: In-Context, Interleaved, and Interactive Any-to-Any Generationcited by 63
  36. 2025VisualCloze: A Universal Image Generation Framework via Visual In-Context Learningcited by 39
  37. 2024VL-ICL Bench: The Devil in the Details of Multimodal In-Context Learningcited by 36
  38. 2023Can Multimodal Large Language Models Truly Perform Multimodal In-Context Learning?cited by 34

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2024From Introspection to Best Practices: Principled Analysis of Demonstrations in Multimodal In-Context Learning
  2. 2026Why Multimodal In-Context Learning Lags Behind? Unveiling the Inner Mechanisms and Bottlenecks
  3. 2025M2IV: Towards Efficient and Fine-grained Multimodal In-Context Learning via Representation Engineering
  4. 2026Enhancing Multimodal In-Context Learning via Inductive-Deductive Reasoning
  5. 2025What do vision-language models see in the context? Investigating multimodal in-context learning
  6. 2026HIFICL: High-Fidelity In-Context Learning for Multimodal Tasks
  7. 2025Advancing Multimodal In-Context Learning in Large Vision-Language Models with Task-aware Demonstrations
  8. 2023Beyond Task Performance: Evaluating and Reducing the Flaws of Large Multimodal Models with In-Context Learning