FractalBench: Diagnosing Visual-Mathematical Reasoning Through Recursive Program Synthesis
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
Mathematical reasoning requires abstracting symbolic rules from visual patterns -- inferring the infinite from the finite. We investigate whether multimodal AI systems possess this capability through FractalBench, a benchmark evaluating fractal program synthesis from images. Fractals provide ideal test cases: Iterated Function Systems with only a few contraction maps generate complex self-similar patterns through simple recursive rules, requiring models to bridge visual perception with mathematical abstraction. We evaluate four leading MLLMs -- GPT-4o, Claude 3.7 Sonnet, Gemini 2.5 Flash, and Qwen 2.5-VL -- on 12 canonical fractals. Models must generate executable Python code reproducing the fractal, enabling objective evaluation. Results reveal a striking disconnect: 76% generate syntactically valid code but only 4% capture mathematical structure. Success varies systematically -- models handle geometric transformations (Koch curves: 17-21%) but fail at branching recursion (trees: <2%), revealing fundamental gaps in mathematical abstraction. FractalBench provides a contamination-resistant diagnostic for visual-mathematical reasoning and is available at https://github.com/NaiveNeuron/FractalBench
Key facts. No citing papers in the map yet (data through 13 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 15 of its 23 references that are in the map. First posted on arXiv 9 Nov 2025.
Builds on
15 of 23 references.
Closest ancestors · 3
Same direction, newest first
- 2025Math Blind: Failures in Diagram Understanding Undermine Reasoning in MLLMs26 Mar
- 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset22 Feb
- 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct
Other references · 12
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,376
- 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,362
- 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 687
- 2020CodeBLEU: a Method for Automatic Evaluation of Code Synthesiscited by 475
- 2016FractalNet: Ultra-Deep Neural Networks without Residualscited by 216
- 2023Structured Chain-of-Thought Prompting for Code Generationcited by 137
- 2024Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Modelscited by 111
- 2024Mind Your Step (by Step): Chain-of-Thought can Reduce Performance on Tasks where Thinking Makes Humans Worsecited by 107
- 2024ErrorRadar: Benchmarking Complex Mathematical Reasoning of Multimodal Large Language Models Via Error Detectioncited by 37
- 2025GeoGramBench: Benchmarking the Geometric Program Reasoning in Modern LLMscited by 13
- 2024What Makes a Maze Look Like a Maze?cited by 11
- 2024TurtleBench: A Visual Programming Benchmark in Turtle Geometrycited by 4
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026M3-ACE: Rectifying Visual Perception in Multimodal Math Reasoning via Multi-Agentic Context Engineering
- 2025VisioMath: Benchmarking Figure-based Mathematical Reasoning in LMMs
- 2025MathSight: A Benchmark Exploring Have Vision-Language Models Really Seen in University-Level Mathematical Reasoning?
- 2025MathOPEval: A Fine-grained Evaluation Benchmark for Visual Operations of MLLMs in Mathematical Reasoning
- 2025Benchmarking Multimodal Mathematical Reasoning with Explicit Visual Dependency
- 2025MathFlow: Enhancing the Perceptual Flow of MLLMs for Visual Mathematical Problems
- 2025MINT-CoT: Enabling Interleaved Visual Tokens in Mathematical Chain-of-Thought Reasoning
- 2026VisTIRA: Closing the Image-Text Modality Gap in Visual Math Reasoning via Structured Tool Integration