mlsanityData to 14 Oct 2026

Automated Model Discovery via Multi-modal & Multi-step Pipeline

arXiv 2509.25946 ↗First posted 30 Sep 2025cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

Automated model discovery is the process of automatically searching and identifying the most appropriate model for a given dataset over a large combinatorial search space. Existing approaches, however, often face challenges in balancing the capture of fine-grained details with ensuring generalizability beyond training data regimes with a reasonable model complexity. In this paper, we present a multi-modal & multi-step pipeline for effective automated model discovery. Our approach leverages two vision-language-based modules (VLM), AnalyzerVLM and EvaluatorVLM, for effective model proposal and evaluation in an agentic way. AnalyzerVLM autonomously plans and executes multi-step analyses to propose effective candidate models. EvaluatorVLM assesses the candidate models both quantitatively and perceptually, regarding the fitness for local details and the generalibility for overall trends. Our results demonstrate that our pipeline effectively discovers models that capture fine details and ensure strong generalizability. Additionally, extensive ablation studies show that both multi-modality and multi-step reasoning play crucial roles in discovering favorable models.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 39 of its 67 references that are in the map. First posted on arXiv 30 Sep 2025.

Builds on

39 of 67 references.

References · 39

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  5. 2025Qwen2.5-VL Technical Reportcited by 5,587
  6. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  7. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  8. 2024A Survey on LLM-as-a-Judgecited by 1,423
  9. 2023Reasoning with Language Model is Planning with World Modelcited by 861
  10. 2019AI Feynman: a Physics-Inspired Method for Symbolic Regressioncited by 548
  11. 2019The frontier of simulation-based inferencecited by 502
  12. 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 500
27 more
  1. 2024MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmarkcited by 372
  2. 2022End-to-end symbolic regression with transformerscited by 203
  3. 2021Neural Symbolic Regression that Scalescited by 192
  4. 2024LLM-SR: Scientific Equation Discovery via Programming with Large Language Modelscited by 133
  5. 2024Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizingcited by 125
  6. 2024DiscoveryBench: Towards Data-Driven Discovery with Large Language Modelscited by 109
  7. 2013Structure Discovery in Nonparametric Regression through Compositional Kernel Searchcited by 88
  8. 2024LLM and Simulation as Bilevel Optimizers: A New Paradigm to Advance Physical Scientific Discoverycited by 86
  9. 2024Symbolic Regression with a Learned Concept Librarycited by 83
  10. 2023Transformer-based Planning for Symbolic Regressioncited by 79
  11. 2022Bayesian Model Selection, the Marginal Likelihood, and Generalizationcited by 57
  12. 2023Deep symbolic regression for physics guided by units constraints: toward the automated discovery of physical lawscited by 55
  13. 2024Automated Statistical Model Discovery with Language Modelscited by 41
  14. 2014Automatic Construction and Natural-Language Description of Nonparametric Regression Modelscited by 40
  15. 2024WaitGPT: Monitoring and Steering Conversational LLM Agent in Data Analysis with On-the-Fly Code Visualizationcited by 39
  16. 2020Probabilistic Grammars for Equation Discoverycited by 36
  17. 2024In-Context Symbolic Regression: Leveraging Large Language Models for Function Discoverycited by 31
  18. 2024Are Large Language Models Good Statisticians?cited by 27
  19. 2024A Survey on Large Language Model-based Agents for Statistics and Data Sciencecited by 22
  20. 2017Scaling up the Automatic Statistician: Scalable Structure Discovery using Gaussian Processescited by 19
  21. 2019Bayesian Synthesis of Probabilistic Programs for Automatic Data Modelingcited by 19
  22. 2022Additive Gaussian Processes Revisitedcited by 19
  23. 2024Evolving Interpretable Visual Classifiers with Large Language Modelscited by 18
  24. 2025Is Your Video Language Model a Reliable Judge?cited by 14
  25. 2022Structural Kernel Search via Bayesian Optimization and Symbolical Optimal Transportcited by 8
  26. 2024In Context Learning and Reasoning for Symbolic Regression with Large Language Modelscited by 7
  27. 2016Time Series Structure Discovery via Probabilistic Program Synthesiscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Aligning What Vision-Language Models See and Perceive with Adaptive Information Flowcited by 3
  2. 2026SVHalluc: Benchmarking Speech-Vision Hallucination in Audio-Visual Large Language Modelscited by 2
  3. 2026VESTA: Visual Exploration with Statistical Tool Agentscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025Mordal: Automated Pretrained Model Selection for Vision Language Models
  2. 2023InfiMM-Eval: Complex Open-Ended Reasoning Evaluation For Multi-Modal Large Language Models
  3. 2026MM-Zero: Self-Evolving Multi-Model Vision Language Models From Zero Data
  4. 2026Rethinking Model Efficiency: Multi-Agent Inference with Large Models
  5. 2026A Probabilistic Framework for LLM-Based Model Discovery
  6. 2024Towards Automated Machine Learning Research
  7. 2025Deploying Tiny LVLM Judges for Real-World Evaluation of Chart Models: Lessons Learned and Best Practices
  8. 2025Meta-Learning and Synthetic Data for Automated Pretraining and Finetuning