Mimicking the Physicist's Eye:A VLM-centric Approach for Physics Formula Discovery
Not on the map. Why?
Abstract (arXiv)
Automated discovery of physical laws from observational data in the real world is a grand challenge in AI. Current methods, relying on symbolic regression or LLMs, are limited to uni-modal data and overlook the rich, visual phenomenological representations of motion that are indispensable to physicists. This "sensory deprivation" severely weakens their ability to interpret the inherent spatio-temporal patterns within dynamic phenomena. To address this gap, we propose VIPER-R1, a multimodal model that performs Visual Induction for Physics-based Equation Reasoning to discover fundamental symbolic formulas. It integrates visual perception, trajectory data, and symbolic reasoning to emulate the scientific discovery process. The model is trained via a curriculum of Motion Structure Induction (MSI), using supervised fine-tuning to interpret kinematic phase portraits and to construct hypotheses guided by a Causal Chain of Thought (C-CoT), followed by Reward-Guided Symbolic Calibration (RGSC) to refine the formula structure with reinforcement learning. During inference, the trained VIPER-R1 acts as an agent: it first posits a high-confidence symbolic ansatz, then proactively invokes an external symbolic regression tool to perform Symbolic Residual Realignment (SR^2). This final step, analogous to a physicist's perturbation analysis, reconciles the theoretical model with empirical data. To support this research, we introduce PhysSymbol, a new 5,000-instance multimodal corpus. Experiments show that VIPER-R1 consistently outperforms state-of-the-art VLM baselines in accuracy and interpretability, enabling more precise discovery of physical laws. Project page: https://jiaaqiliu.github.io/VIPER-R1/
Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 35 of its 48 references that are in the map. First posted on arXiv 24 Aug 2025.
Builds on
35 of 48 references.
References · 35
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
- 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,919
- 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,071
- 2023Vision-Language Models for Vision Tasks: A Surveycited by 839
- 2024GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Modelscited by 550
- 2019AI Feynman: a Physics-Inspired Method for Symbolic Regressioncited by 548
- 2019Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradientscited by 309
- 2023Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Taskscited by 246
- 2021Contemporary Symbolic Regression Methods and their Relative Performancecited by 215
23 more
- 2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontierscited by 199
- 2021Neural Symbolic Regression that Scalescited by 192
- 2023SciMON: Scientific Inspiration Machines Optimized for Noveltycited by 179
- 2022Evolution through Large Modelscited by 134
- 2024LLM-SR: Scientific Equation Discovery via Programming with Large Language Modelscited by 133
- 2023Hypothesis Search: Inductive Reasoning with Language Modelscited by 120
- 2024DiscoveryBench: Towards Data-Driven Discovery with Large Language Modelscited by 109
- 2024A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discoverycited by 96
- 2024LLM and Simulation as Bilevel Optimizers: A New Paradigm to Advance Physical Scientific Discoverycited by 86
- 2024Symbolic Regression with a Learned Concept Librarycited by 83
- 2023Can GPT-4 Perform Neural Architecture Search?cited by 74
- 2022Symbolic Regression is NP-hardcited by 71
- 2025LLM-SRBench: A New Benchmark for Scientific Equation Discovery with Large Language Modelscited by 71
- 2022Symbolic Physics Learner: Discovering governing equations via Monte Carlo tree searchcited by 66
- 2024Large Language Models As Evolution Strategiescited by 58
- 2023SNIP: Bridging Mathematical Symbolic and Numeric Realms with Unified Pre-trainingcited by 47
- 2022Rethinking Symbolic Regression Datasets and Benchmarks for Scientific Discoverycited by 46
- 2023Large Language Models are Zero Shot Hypothesis Proposerscited by 45
- 2024Automated Statistical Model Discovery with Language Modelscited by 41
- 2024Towards Scientific Discovery with Generative AI: Progress, Opportunities, and Challengescited by 41
- 2023Large Language Models for Scientific Synthesis, Inference and Explanationcited by 39
- 2024Data-driven Discovery with Large Generative Modelscited by 34
- 2025MLLM-based Discovery of Intrinsic Coordinates and Governing Equations from High-Dimensional Datacited by 2
Built on by
Citing papers per year: 2025: 3; 2026: 4.
7 papers.
- 2025NewtonBench: Benchmarking Generalizable Scientific Law Discovery in LLM Agentscited by 36
- 2025Agent0-VL: Exploring Self-Evolving Agent for Tool-Integrated Vision-Language Reasoningcited by 31
- 2026IRIS: A Real-World Benchmark for Inverse Recovery and Identification of Physical Dynamic Systems from Monocular Videocited by 4
- 2025CircuitSense: A Hierarchical MLLM Benchmark Bridging Visual Comprehension and Symbolic Reasoning in Engineering Design Processcited by 3
- 2026MATRIX: A Multimodal Benchmark and Post-Training Framework for Materials Sciencecited by 3
- 2026Scaling Participation in Modular AI Systemscited by 2
- 2026Hidden in Plain Sight: Visual-to-Symbolic Analytical Solution Inference from Field Visualizationscited by 0
- 2026Scaling Participation in Modular AI Systems5 Jun 2026
- 2026Hidden in Plain Sight: Visual-to-Symbolic Analytical Solution Inference from Field Visualizations10 Apr 2026
- 2026IRIS: A Real-World Benchmark for Inverse Recovery and Identification of Physical Dynamic Systems from Monocular Video17 Mar 2026
- 2026MATRIX: A Multimodal Benchmark and Post-Training Framework for Materials Science30 Jan 2026
- 2025Agent0-VL: Exploring Self-Evolving Agent for Tool-Integrated Vision-Language Reasoning25 Nov 2025
- 2025NewtonBench: Benchmarking Generalizable Scientific Law Discovery in LLM Agents8 Oct 2025
- 2025CircuitSense: A Hierarchical MLLM Benchmark Bridging Visual Comprehension and Symbolic Reasoning in Engineering Design Process26 Sep 2025
Similar, not cited
Close in wording, no citation either way
- 2026P1-VL: Bridging Visual Perception and Scientific Reasoning in Physics Olympiads
- 2026Think like a Scientist: Physics-guided LLM Agent for Equation Discovery
- 2025Aligning Perception, Reasoning, Modeling and Interaction: A Survey on Physical AI
- 2026Towards Physics-Faithful Generation of Scientific Diagrams
- 2026Decoupled Physical Modeling and Execution for Physics Reasoning
- 2026DiscoverPhysics: Benchmarking LLMs for Out-of-the-Box Scientific Thinking
- 2026Causal Physics Steering in Video World Models via Concept Activation Vectors
- 2026PhysicsMind: Sim and Real Mechanics Benchmarking for Physical Reasoning and Prediction in Foundational VLMs and World Models