# No Foundations without Foundations -- Why semi-mechanistic models are essential for regulatory biology

arXiv: https://arxiv.org/abs/2501.19178 · HTML: https://mlsanity.com/paper/2501.19178/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 75 references that are in the map.
- First posted on arXiv 31 Jan 2025.

## Builds on (most cited first, 17 of 17)

- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [On Neural Differential Equations](https://mlsanity.com/paper/2202.02435/) (2022)
- [Permutation-based Causal Inference Algorithms with Interventions](https://mlsanity.com/paper/1705.10220/) (2017)
- [Characterizing and Learning Equivalence Classes of Causal DAGs under Interventions](https://mlsanity.com/paper/1802.06310/) (2018)
- [PerturBench: Benchmarking Machine Learning Models for Cellular Perturbation Analysis](https://mlsanity.com/paper/2408.10609/) (2024)
- [Learning Causal Representations of Single Cells via Sparse Mechanism Shift Modeling](https://mlsanity.com/paper/2211.03553/) (2022)
- [Active Learning for Optimal Intervention Design in Causal Models](https://mlsanity.com/paper/2209.04744/) (2022)
- [Deep Learning of Causal Structures in High Dimensions](https://mlsanity.com/paper/2212.04866/) (2022)
- [Causal Modeling with Stationary Diffusions](https://mlsanity.com/paper/2310.17405/) (2023)
- [Benchmarking Transcriptomics Foundation Models for Perturbation Analysis : one PCA still rules them all](https://mlsanity.com/paper/2410.13956/) (2024)
- [Near-Optimal Multi-Perturbation Experimental Design for Causal Structure Learning](https://mlsanity.com/paper/2105.14024/) (2021)
- [Season combinatorial intervention predictions with Salt & Peper](https://mlsanity.com/paper/2404.16907/) (2024)
- [Learning Identifiable Factorized Causal Representations of Cellular Responses](https://mlsanity.com/paper/2410.22472/) (2024)
- [DiscoGen: Learning to Discover Gene Regulatory Networks](https://mlsanity.com/paper/2304.05823/) (2023)
- [Simulation-based Benchmarking for Causal Structure Learning in Gene Perturbation Experiments](https://mlsanity.com/paper/2407.06015/) (2024)
- [PyRelationAL: a python library for active learning research and development](https://mlsanity.com/paper/2205.11117/) (2022)
- [Joint Causal Inference from Multiple Contexts](https://mlsanity.com/paper/1611.10351/) (2016)

## Built on by (most cited first, 3 of 3)

- [Chem-PerturBridge: a harmonized compendium of small molecule perturbation transcriptomic effects](https://mlsanity.com/paper/2605.31522/) (2026)
- [When three experiments are better than two: Avoiding intractable correlated aleatoric uncertainty by leveraging a novel bias--variance tradeoff](https://mlsanity.com/paper/2509.04363/) (2025)
- [An Investigation of the NeurIPS and ICML 2025 Position Tracks](https://mlsanity.com/paper/2608.16894/) (2026)

## Abstract (arXiv)

Despite substantial efforts, deep learning has not yet delivered a transformative impact on elucidating regulatory biology, particularly in the realm of predicting gene expression profiles. Here, we argue that genuine "foundation models" of regulatory biology will remain out of reach unless guided by frameworks that integrate mechanistic insight with principled experimental design. We present one such ground-up, semi-mechanistic framework that unifies perturbation-based experimental designs across both in vitro and in vivo CRISPR screens, accounting for differentiating and non-differentiating cellular systems. By revealing previously unrecognised assumptions in published machine learning methods, our approach clarifies links with popular techniques such as variational autoencoders and structural causal models. In practice, this framework suggests a modified loss function that we demonstrate can improve predictive performance, and further suggests an error analysis that informs batching strategies. Ultimately, since cellular regulation emerges from innumerable interactions amongst largely uncharted molecular components, we contend that systems-level understanding cannot be achieved through structural biology alone. Instead, we argue that real progress will require a first-principles perspective on how experiments capture biological phenomena, how data are generated, and how these processes can be reflected in more faithful modelling architectures.