mlsanityData to 14 Oct 2026

MathSticks: A Benchmark for Visual Symbolic Compositional Reasoning with Matchstick Puzzles

arXiv 2510.00483 ↗First posted 1 Oct 2025cs.CVcited by 5

On the map: LLM reasoning → Multimodal reasoning

Abstract (arXiv)

We introduce textscMathSticks, a benchmark for Visual Symbolic Compositional Reasoning (VSCR), which unifies visual perception, symbolic manipulation, and arithmetic consistency. Each task presents an incorrect matchstick equation that must be corrected by moving one or two sticks under strict conservation rules. The benchmark includes both text-guided and purely visual settings, systematically covering digit scale, move complexity, solution multiplicity, and operator variation, with 1.4M generated instances and a curated test set. Evaluations of 14 vision--language models reveal substantial limitations: closed-source models succeed only on simple cases, open-source models fail in the visual regime, while humans exceed 90% accuracy. These findings establish textscMathSticks as a rigorous testbed for advancing compositional reasoning across vision and symbols. Our code and dataset are publicly available at https://github.com/Yuheng2000/MathSticks.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 29 of its 64 references that are in the map. First posted on arXiv 1 Oct 2025.

Builds on

29 of 64 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Reason-RFT: Reinforcement Fine-Tuning for Visual Reasoning of Vision Language Models26 Mar
  2. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling6 Dec
  3. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct

Other references · 26

  1. 2025Qwen2.5-VL Technical Reportcited by 5,587
  2. 2024GPT-4o System Cardcited by 4,588
  3. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  4. 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 881
  5. 2025HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Modelcited by 181
  6. 2025Seed1.5-Thinking: Advancing Superb Reasoning Models with Reinforcement Learningcited by 161
  7. 2025RoboRefer: Towards Spatial Referring with Reasoning in Vision-Language Models for Roboticscited by 127
  8. 2024Draw-and-Understand: Leveraging Visual Prompts to Enable MLLMs to Comprehend What You Wantcited by 101
  9. 2025Beyond Attention or Similarity: Maximizing Conditional Diversity for Token Pruning in MLLMscited by 96
  10. 2025RoboBrain 2.0 Technical Reportcited by 89
  11. 2024Code-as-Monitor: Constraint-aware Visual Programming for Reactive and Proactive Robotic Failure Detectioncited by 63
  12. 2024LaMP: Language-Motion Pretraining for Motion Generation, Retrieval, and Captioningcited by 59
14 more
  1. 2024MineDreamer: Learning to Follow Instructions via Chain-of-Imagination for Simulated-World Controlcited by 53
  2. 2025RoboOS: A Hierarchical Embodied Framework for Cross-Embodiment and Multi-Agent Collaborationcited by 28
  3. 2025EmbodiedOcc++: Boosting Embodied 3D Occupancy Prediction with Plane Regularization and Uncertainty Samplercited by 27
  4. 2025ManipLVM-R1: Reinforcement Learning for Reasoning in Embodied Manipulation with Large Vision-Language Modelscited by 27
  5. 2024RH20T-P: A Primitive-Level Robotic Dataset Towards Composable Generalization Agentscited by 20
  6. 2025MSC-Bench: Benchmarking and Analyzing Multi-Sensor Corruption for Driving Perceptioncited by 15
  7. 2025VisualTrans: A Benchmark for Real-World Visual Transformation Reasoningcited by 12
  8. 2025ManipDreamer3D : Synthesizing Plausible Robotic Manipulation Video with Occupancy-aware 3D Trajectorycited by 12
  9. 2023General Point Model with Autoencoding and Autoregressivecited by 9
  10. 2025What Really Matters for Robust Multi-Sensor HD Map Construction?cited by 9
  11. 2025EgoPrompt: Prompt Learning for Egocentric Action Recognitioncited by 4
  12. 2023Visual Transformation Tellingcited by 3
  13. 2025FastRSR: Efficient and Accurate Road Surface Reconstruction from Bird's Eye Viewcited by 3
  14. 2024MulSMo: Multimodal Stylized Motion Generation by Bidirectional Control Flowcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 2.

5 papers.

  1. 2025TIGeR: Tool-Integrated Geometric Reasoning in Vision-Language Models for Roboticscited by 27
  2. 2025Towards Cross-View Point Correspondence in Vision-Language Modelscited by 14
  3. 2025Scaling Up AI-Generated Image Detection with Generator-Aware Prototypescited by 9
  4. 2026PReD: An LLM-based Foundation Multimodal Model for Electromagnetic Perception, Recognition, and Decisioncited by 2
  5. 2026SARFusion: Scene-Aware Routing Fusion for Robust Camera-LiDAR 3D Object Detectioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025MathSight: A Benchmark Exploring Have Vision-Language Models Really Seen in University-Level Mathematical Reasoning?
  2. 2025Can Vision-Language Models Solve Visual Math Equations?
  3. 2025VisioMath: Benchmarking Figure-based Mathematical Reasoning in LMMs
  4. 2024VisAidMath: Benchmarking Visual-Aided Mathematical Reasoning
  5. 2022Abstract Visual Reasoning with Tangram Shapes
  6. 2024Take A Step Back: Rethinking the Two Stages in Visual Reasoning
  7. 2026Reasoning or Pattern Matching? Probing Large Vision-Language Models with Visual Puzzles
  8. 2025MINT-CoT: Enabling Interleaved Visual Tokens in Mathematical Chain-of-Thought Reasoning