mlsanityData to 7 Oct 2026

Show Me Examples: Inferring Visual Concepts from Image Sets

arXiv 2607.02402 ↗First posted 2 Jul 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Vision-language models (VLMs) can follow complex textual instructions, yet they struggle to reason from purely visual context. In particular, current models fail to infer shared concepts from sets of example images and apply them to new inputs. We introduce Visual Concept Inference from Sets (VICIS), a task that evaluates this capability. Given a small context set of images sharing a concept and a query image, the model must generate new images that preserve the context-defined concept while remaining consistent with the query. We show that state-of-the-art VLMs perform poorly on this task, often ignoring the visual context or defaulting to biased generations. To address this gap, we propose a training framework and architecture that learn to infer visual concepts from image sets and extract concept-specific embeddings from queries. Experiments on synthetic data and large-scale ImageNet/WordNet data show that our model generates more accurate and diverse outputs and generalizes to unseen concepts and modalities such as sketches.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 34 of its 41 references that are in the map. First posted on arXiv 2 Jul 2026.

Builds on

34 of 41 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Flamingo: a Visual Language Model for Few-Shot Learning29 Apr

Other references · 33

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2025Qwen3 Technical Reportcited by 8,195
  6. 2023Visual Instruction Tuningcited by 7,792
  7. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  8. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  9. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  10. 2022Flow Matching for Generative Modelingcited by 5,426
  11. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  12. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
21 more
  1. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,010
  2. 2025Qwen-Image Technical Reportcited by 973
  3. 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 960
  4. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 929
  5. 2023Stochastic Interpolants: A Unifying Framework for Flows and Diffusionscited by 897
  6. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
  7. 2023Vision Transformers Need Registerscited by 844
  8. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 787
  9. 2019Variational Autoencoders and Nonlinear ICA: A Unifying Frameworkcited by 494
  10. 2022Visual Prompting via Image Inpaintingcited by 248
  11. 2023Sequential Modeling Enables Scalable Learning for Large Vision Modelscited by 212
  12. 2019GANalyze: Toward Visual Definitions of Cognitive Image Propertiescited by 193
  13. 2025Do generative video models understand physical principles?cited by 151
  14. 2023Concept Sliders: LoRA Adaptors for Precise Control in Diffusion Modelscited by 148
  15. 2025ILLUME+: Illuminating Unified MLLM with Dual Visual Tokenization and Diffusion Refinementcited by 53
  16. 2020CausalVAE: Structured Causal Disentanglement in Variational Autoencodercited by 47
  17. 2023Towards In-context Scene Understandingcited by 45
  18. 2024Continuous, Subject-Specific Attribute Control in T2I Models by Identifying Semantic Directionscited by 39
  19. 2025RelationAdapter: Learning and Transferring Visual Relation with Diffusion Transformerscited by 26
  20. 2025SliderSpace: Decomposing the Visual Capabilities of Diffusion Modelscited by 24
  21. 2023IMProv: Inpainting-based Multimodal Prompting for Computer Vision Taskscited by 10

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026VisualScratchpad: Inference-time Visual Concepts Analysis in Vision Language Models
  2. 2024Understanding Visual Concepts Across Models
  3. 2023Cross-Modal Concept Learning and Inference for Vision-Language Models
  4. 2023Probing Conceptual Understanding of Large Visual-Language Models
  5. 2026Extraction and Analysis of Multimodal Concepts in Vision Language Models through Sparse Autoencoders
  6. 2023Language-Informed Visual Concept Learning
  7. 2024Visual In-Context Learning for Large Vision-Language Models
  8. 2025Bridging the gap to real-world language-grounded visual concept learning