# DiSciPLE: Learning Interpretable Programs for Scientific Visual Discovery

arXiv: https://arxiv.org/abs/2502.10060 · HTML: https://mlsanity.com/paper/2502.10060/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 47 references that are in the map.
- First posted on arXiv 14 Feb 2025.

## Builds on (most cited first, 30 of 33)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Code as Policies: Language Model Programs for Embodied Control](https://mlsanity.com/paper/2209.07753/) (2022)
- [Concept Bottleneck Models](https://mlsanity.com/paper/2007.04612/) (2020)
- [ViperGPT: Visual Inference via Python Execution for Reasoning](https://mlsanity.com/paper/2303.08128/) (2023)
- [Visual Programming: Compositional visual reasoning without training](https://mlsanity.com/paper/2211.11559/) (2022)
- [The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervision](https://mlsanity.com/paper/1904.12584/) (2019)
- [Visual Classification via Description from Large Language Models](https://mlsanity.com/paper/2210.07183/) (2022)
- [Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classification](https://mlsanity.com/paper/2211.11158/) (2022)
- [Label-Free Concept Bottleneck Models](https://mlsanity.com/paper/2304.06129/) (2023)
- [What does a platypus look like? Generating customized prompts for zero-shot image classification](https://mlsanity.com/paper/2209.03320/) (2022)
- [Evaluation of Output Embeddings for Fine-Grained Image Classification](https://mlsanity.com/paper/1409.8403/) (2014)
- [BioCLIP: A Vision Foundation Model for the Tree of Life](https://mlsanity.com/paper/2311.18803/) (2023)
- [Neural Logic Machines](https://mlsanity.com/paper/1904.11694/) (2019)
- [Semantic Autoencoder for Zero-Shot Learning](https://mlsanity.com/paper/1704.08345/) (2017)
- [LLM-SR: Scientific Equation Discovery via Programming with Large Language Models](https://mlsanity.com/paper/2404.18400/) (2024)
- [Inferring and Executing Programs for Visual Reasoning](https://mlsanity.com/paper/1705.03633/) (2017)
- [Interpretable Scientific Discovery with Symbolic Regression: A Review](https://mlsanity.com/paper/2211.10873/) (2022)
- [Symbolic Regression with a Learned Concept Library](https://mlsanity.com/paper/2409.09359/) (2024)
- [Remote Sensing Vision-Language Foundation Models without Annotations via Ground Remote Alignment](https://mlsanity.com/paper/2312.06960/) (2023)
- [Part-Stacked CNN for Fine-Grained Visual Categorization](https://mlsanity.com/paper/1512.08086/) (2015)
- [SustainBench: Benchmarks for Monitoring the Sustainable Development Goals with Machine Learning](https://mlsanity.com/paper/2111.04724/) (2021)
- [Automated Statistical Model Discovery with Language Models](https://mlsanity.com/paper/2402.17879/) (2024)
- [Revisiting Pose-Normalization for Fine-Grained Few-Shot Recognition](https://mlsanity.com/paper/2004.00705/) (2020)
- [In-Context Symbolic Regression: Leveraging Large Language Models for Function Discovery](https://mlsanity.com/paper/2404.19094/) (2024)
- [OpenStreetMap: Challenges and Opportunities in Machine Learning and Remote Sensing](https://mlsanity.com/paper/2007.06277/) (2020)
- [Evolving Interpretable Visual Classifiers with Large Language Models](https://mlsanity.com/paper/2404.09941/) (2024)
- [MuseCL: Predicting Urban Socioeconomic Indicators via Multi-Semantic Contrastive Learning](https://mlsanity.com/paper/2407.09523/) (2024)
- [LLMs as Visual Explainers: Advancing Image Classification with Evolving Visual Descriptions](https://mlsanity.com/paper/2311.11904/) (2023)

## Built on by (most cited first, 3 of 3)

- [VDAWorld: World Modelling via VLM-Directed Abstraction and Simulation](https://mlsanity.com/paper/2512.11061/) (2025)
- [MaskDiME: Adaptive Masked Diffusion for Precise and Efficient Visual Counterfactual Explanations](https://mlsanity.com/paper/2602.18792/) (2026)
- [Teaching Humans Subtle Differences with DIFFusion](https://mlsanity.com/paper/2504.08046/) (2025)

## Abstract (arXiv)

Visual data is used in numerous different scientific workflows ranging from remote sensing to ecology. As the amount of observation data increases, the challenge is not just to make accurate predictions but also to understand the underlying mechanisms for those predictions. Good interpretation is important in scientific workflows, as it allows for better decision-making by providing insights into the data. This paper introduces an automatic way of obtaining such interpretable-by-design models, by learning programs that interleave neural networks. We propose DiSciPLE (Discovering Scientific Programs using LLMs and Evolution) an evolutionary algorithm that leverages common sense and prior knowledge of large language models (LLMs) to create Python programs explaining visual data. Additionally, we propose two improvements: a program critic and a program simplifier to improve our method further to synthesize good programs. On three different real-world problems, DiSciPLE learns state-of-the-art programs on novel tasks with no prior literature. For example, we can learn programs with 35% lower error than the closest non-interpretable baseline for population density estimation.