Exemplar2VQA: A Scalable Exemplar-Driven Visual Question Answering Generation Framework via Multi-Agent Coding
On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models
also in 1 more
Abstract (arXiv)
Advancing spatial intelligence in Multimodal Large Language Models (MLLMs) is bottlenecked by the scarcity of complex, scalable 3D question-answer (QA) data. While manual annotation is labor-intensive, directly utilizing LLMs to synthesize these QA pairs often fails due to their inherent deficiencies in spatial and geometric computation. We introduce Exemplar2VQA, a scalable exemplar-driven visual question answering generation framework that rapidly synthesizes large-scale spatial QA pairs in simulated environments via multi-agent coding. By equipping collaborative agents with a meticulously designed library of geometric utilities, Exemplar2VQA bypasses LLMs' spatial reasoning flaws through deterministic code execution. Crucially, the framework exhibits remarkable versatility: taking diverse static object-centric spatial query templates as exemplars, it seamlessly and autonomously scales them into massive, high-fidelity synthetic datasets. Fine-tuning Qwen2.5-VL (3B/7B) exclusively on Exemplar2VQA-generated synthetic indoor data yields significant performance improvements across various diverse benchmarks. Furthermore, its effectiveness is not limited to in-domain indoor datasets but also robustly extends to outdoor and mixed-scene benchmarks. These results establish Exemplar2VQA as a scalable and powerful paradigm for bridging the sim-to-real gap in Embodied AI. Our code is at https://github.com/yingjiayu12/Exemplar2VQA
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 49 of its 56 references that are in the map. First posted on arXiv 29 Sep 2026.
Builds on
49 of 56 references.
Closest ancestors · 10
Same direction, newest first
- 2026pySpatial: Generating 3D Visual Programs for Zero-Shot Spatial Reasoning1 Mar
- 2025SpaCE-10: A Comprehensive Benchmark for Multimodal Large Language Models in Compositional Spatial Intelligence9 Jun
- 2025MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligence29 May
- 2025ViewSpatial-Bench: Evaluating Multi-perspective Spatial Localization in Vision-Language Models27 May
- 2025Multi-SpatialMLLM: Multi-Frame Spatial Understanding with Multi-Modal Large Language Models22 May
- 2025Are Multimodal Large Language Models Ready for Omnidirectional Spatial Reasoning?17 May
- 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces18 Dec
- 20243D Spatial Understanding in MLLMs: Disambiguation and Evaluation9 Dec
- 2024An Empirical Analysis on Spatial Reasoning Capabilities of Large Multimodal Models9 Nov
- 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities22 Jan
Other references · 39
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2025Qwen3 Technical Reportcited by 8,820
- 2023Visual Instruction Tuningcited by 7,869
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2024GPT-4o System Cardcited by 4,588
- 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
- 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
- 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
- 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
27 more
- 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,808
- 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,729
- 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
- 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
- 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,457
- 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,332
- 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,210
- 2019Habitat: A Platform for Embodied AI Researchcited by 1,191
- 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,076
- 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,050
- 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,005
- 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 869
- 2023ViperGPT: Visual Inference via Python Execution for Reasoningcited by 603
- 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Modelscited by 583
- 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 583
- 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 572
- 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 550
- 2022Visual Programming: Compositional visual reasoning without trainingcited by 516
- 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 380
- 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 346
- 2022Visual Spatial Reasoningcited by 293
- 2023Holodeck: Language Guided Generation of 3D Embodied AI Environmentscited by 268
- 2024MuirBench: A Comprehensive Benchmark for Robust Multi-image Understandingcited by 188
- 2024CodeACT: Code Adaptive Compute-efficient Tuning Framework for Code LLMscited by 15
- 2024Towards Foundation Models for 3D Vision: How Close Are We?cited by 13
- 2025HyCoRA: Hyper-Contrastive Role-Adaptive Learning for Role-Playingcited by 7
- 2025FLEG: Feed-Forward Language Embedded Gaussian Splatting from Any Views via Compact Semantic Representationcited by 4
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025VQ-VA World: Towards High-Quality Visual Question-Visual Answering
- 2024Multi-Agents Based on Large Language Models for Knowledge-based Visual Question Answering
- 2025OWLViz: An Open-World Benchmark for Visual Question Answering
- 2026FAST-EQA: Efficient Embodied Question Answering with Global and Local Region Relevancy
- 2026Efficient Multimodal Planning Agent for Visual Question-Answering
- 20253D-MoRe: Unified Modal-Contextual Reasoning for Embodied Question Answering
- 2025CityEQA: A Hierarchical LLM Agent on Embodied Question Answering Benchmark in City Space
- 2026ViewMind3D: Modular View-Aware Inference for Training-Free 3D-QA