Visual Set Program Synthesizer
Not on the map. Why?
Abstract (arXiv)
A user pointing their phone at a supermarket shelf and asking "Which soda has the least sugar?" poses a difficult challenge for current visual Al assistants. Such queries require not only object recognition, but explicit set-based reasoning such as filtering, comparison, and aggregation. Standard endto-end MLLMs often fail at these tasks because they lack an explicit mechanism for compositional logic. We propose treating visual reasoning as Visual Program Synthesis, where the model first generates a symbolic program that is executed by a separate engine grounded in visual scenes. We also introduce Set-VQA, a new benchmark designed specifically for evaluating set-based visual reasoning. Experiments show that our approach significantly outperforms state-of-the-art baselines on complex reasoning tasks, producing more systematic and transparent behavior while substantially improving answer accuracy. These results demonstrate that program-driven reasoning provides a principled alternative to black-box visual-language inference.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 20 of its 23 references that are in the map. First posted on arXiv 16 Mar 2026.
Builds on
20 of 23 references.
References · 20
- 2023GPT-4 Technical Reportcited by 18,396
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2023Visual Instruction Tuningcited by 7,869
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 708
- 2023ViperGPT: Visual Inference via Python Execution for Reasoningcited by 603
- 2022Visual Programming: Compositional visual reasoning without trainingcited by 516
- 2024YOLO-World: Real-Time Open-Vocabulary Object Detectioncited by 503
- 2019Scene Text Visual Question Answeringcited by 331
- 2018Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understandingcited by 304
8 more
- 2025Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMscited by 289
- 2025Model Context Protocol (MCP): Landscape, Security Threats, and Future Research Directionscited by 265
- 2025Reinforcement Learning with Verifiable Rewards: GRPO's Effective Loss, Dynamics, and Success Amplificationcited by 110
- 2021PTR: A Benchmark for Part-based Conceptual, Relational, and Physical Reasoningcited by 34
- 2023Neural Task Synthesis for Visual Programmingcited by 13
- 2024Potential-Based Reward Shaping For Intrinsic Motivationcited by 12
- 2019A new Potential-Based Reward Shaping for Reinforcement Learning Agentcited by 8
- 2024Affinity-Graph-Guided Contractive Learning for Pretext-Free Medical Image Segmentation with Minimal Annotationcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024From the Least to the Most: Building a Plug-and-Play Visual Reasoner via Data Synthesis
- 2024PropTest: Automatic Property Testing for Improved Visual Programming
- 2025RECODE: Reasoning Through Code Generation for Visual Question Answering
- 2025On Data Synthesis and Post-training for Visual Abstract Reasoning
- 2023See, Think, Confirm: Interactive Prompting Between Vision and Language Models for Knowledge-based Visual Reasoning
- 2025Synthesizing Visual Concepts as Vision-Language Programs
- 2026Hob-VL: A Benchmark for Visually Grounded Boolean Reasoning
- 2024Zero-Shot Visual Reasoning by Vision-Language Models: Benchmarking and Analysis