SPHINX: A Synthetic Environment for Visual Perception and Reasoning
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
We present Sphinx, a synthetic environment for visual perception and reasoning that targets core cognitive primitives. Sphinx procedurally generates puzzles using motifs, tiles, charts, icons, and geometric primitives, each paired with verifiable ground-truth solutions, enabling both precise evaluation and large-scale dataset construction. The benchmark covers 25 task types spanning symmetry detection, geometric transformations, spatial reasoning, chart interpretation, and sequence prediction. Evaluating recent large vision-language models (LVLMs) shows that even state-of-the-art GPT-5 attains only 51.1% accuracy, well below human performance. Finally, we demonstrate that reinforcement learning with verifiable rewards (RLVR) substantially improves model accuracy on these tasks and yields gains on external visual reasoning benchmarks, highlighting its promise for advancing multimodal reasoning.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 33 of its 74 references that are in the map. First posted on arXiv 25 Nov 2025.
Builds on
33 of 74 references.
Closest ancestors · 11
Same direction, newest first
- 2025ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systems17 May
- 2025VisuLogic: A Benchmark for Evaluating Visual Reasoning in Multi-modal Large Language Models21 Apr
- 2025Reason-RFT: Reinforcement Fine-Tuning for Visual Reasoning of Vision Language Models26 Mar
- 2025V-STaR: Benchmarking Video-LLMs on Video Spatio-Temporal Reasoning14 Mar
- 2025R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalization13 Mar
- 2025LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RL10 Mar
- 2025Visual-RFT: Visual Reinforcement Fine-Tuning3 Mar
- 2024Reasoning Limitations of Multimodal Large Language Models. A Case Study of Bongard Problems2 Nov
- 2024LogicVista: Multimodal LLM Logical Reasoning Benchmark in Visual Contexts6 Jul
- 2024GITA: Graph to Visual and Textual Integration for Vision-Language Graph Reasoning3 Feb
- 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct
Other references · 22
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2025Qwen3 Technical Reportcited by 8,195
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
- 2024OpenAI o1 System Cardcited by 1,220
- 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 948
- 2022When and why vision-language models behave like bags-of-words, and what to do about it?cited by 603
- 2021Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoningcited by 376
- 2025Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRscited by 373
10 more
- 2025Learning to Reason under Off-Policy Guidancecited by 266
- 2025DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and Verifiable Mathematical Dataset for Advancing Reasoningcited by 261
- 2025Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learningcited by 260
- 2024MMT-Bench: A Comprehensive Multimodal Benchmark for Evaluating Large Vision-Language Models Towards Multitask AGIcited by 192
- 2025REASONING GYM: Reasoning Environments for Reinforcement Learning with Verifiable Rewardscited by 121
- 2025Big-Math: A Large-Scale, High-Quality Math Dataset for Reinforcement Learning in Language Modelscited by 104
- 2025Learning What Reinforcement Learning Can't: Interleaved Online Fine-Tuning for Hardest Questionscited by 95
- 2025Enigmata: Scaling Logical Reasoning in Large Language Models with Synthetic Verifiable Puzzlescited by 53
- 2025Jigsaw-R1: A Study of Rule-based Visual Reinforcement Learning with Jigsaw Puzzlescited by 36
- 2024What is the Visual Cognition Gap between Humans and Multimodal LLMs?cited by 30
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026Reasoning or Pattern Matching? Probing Large Vision-Language Models with Visual Puzzles
- 2026Disentangling Perception and Reasoning in Multimodal LLMs via Reward Design
- 2025Zebra-CoT: A Dataset for Interleaved Vision Language Reasoning
- 2025VisuRiddles: Fine-grained Perception is a Primary Bottleneck for Multimodal Large Language Models in Abstract Visual Reasoning
- 2026TACIT Benchmark: A Programmatic Visual Reasoning Benchmark for Generative and Discriminative Models
- 2026VBVR-Pro: A Scalable and Verifiable Suite for Native Visual Reasoning
- 2025DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning
- 2025V-Thinker: Interactive Thinking with Images