mlsanityData to 7 Oct 2026

Discovering Symbolic Models from Deep Learning with Inductive Biases

arXiv 2006.11287 ↗First posted 19 Jun 2020cs.LG astro-ph.CO astro-ph.IMcited by 335

Not on the map. Why?

Abstract (arXiv)

We develop a general approach to distill symbolic representations of a learned deep model by introducing strong inductive biases. We focus on Graph Neural Networks (GNNs). The technique works as follows: we first encourage sparse latent representations when we train a GNN in a supervised setting, then we apply symbolic regression to components of the learned model to extract explicit physical relations. We find the correct known equations, including force laws and Hamiltonians, can be extracted from the neural network. We then apply our method to a non-trivial cosmology example-a detailed dark matter simulation-and discover a new analytic formula which can predict the concentration of dark matter from the mass distribution of nearby cosmic structures. The symbolic expressions extracted from the GNN using our technique also generalized to out-of-distribution data better than the GNN itself. Our approach offers alternative directions for interpreting neural networks and discovering novel physical principles from the representations they learn.

Key facts. Cited by 335 papers in the map (data through 7 Oct 2026). First posted on arXiv 19 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 46; 2022: 64; 2023: 60; 2024: 48; 2025: 51; 2026: 66.

335 papers.

Sort:

By area:

  1. 2022End-to-end symbolic regression with transformerscited by 195
  2. 2021Neural Symbolic Regression that Scalescited by 185
  3. 2022Interpretable and Generalizable Graph Learning via Stochastic Attention Mechanismcited by 171
  4. 2022Forecasting Global Weather with Graph Neural Networkscited by 165
  5. 2021Enhancing Computational Fluid Dynamics with Machine Learningcited by 137
  6. 2024KAN 2.0: Kolmogorov-Arnold Networks Meet Sciencecited by 116
  7. 2022Interpretable Scientific Discovery with Symbolic Regression: A Reviewcited by 105
  8. 2022Physics-Informed Machine Learning: A Survey on Problems, Methods and Applicationscited by 88
  9. 2021Chaos as an interpretable benchmark for forecasting and data-driven modellingcited by 87
  10. 2022SELFIES and the future of molecular string representationscited by 76
  11. 2024Symbolic Regression with a Learned Concept Librarycited by 76
  12. 2024Building Machines that Learn and Think with Peoplecited by 73
38 more
  1. 2023Transformer-based Planning for Symbolic Regressioncited by 72
  2. 2022Rediscovering orbital mechanics with machine learningcited by 71
  3. 2021A Survey on Interpretable Reinforcement Learningcited by 70
  4. 2022On scientific understanding with artificial intelligencecited by 68
  5. 2021Encoding physics to learn reaction-diffusion processescited by 61
  6. 2022Theory of Graph Neural Networks: Representation and Learningcited by 60
  7. 2021Neuro-Symbolic Artificial Intelligence: Current Trendscited by 54
  8. 2022Dissipative Hamiltonian Neural Networks: Learning Dissipative and Conservative Dynamics Separatelycited by 46
  9. 2025What Has a Foundation Model Found? Using Inductive Bias to Probe for World Modelscited by 46
  10. 2022Rethinking Symbolic Regression Datasets and Benchmarks for Scientific Discoverycited by 44
  11. 2023SNIP: Bridging Mathematical Symbolic and Numeric Realms with Unified Pre-trainingcited by 41
  12. 2024Towards Scientific Discovery with Generative AI: Progress, Opportunities, and Challengescited by 41
  13. 2022How Much Data Are Augmentations Worth? An Investigation into Scaling Laws, Invariance, and Implicit Regularizationcited by 40
  14. 2023Deep Generative Symbolic Regression with Monte-Carlo-Tree-Searchcited by 40
  15. 2023Learning Transformer Programscited by 40
  16. 2023Seeing is Believing: Brain-Inspired Modular Training for Mechanistic Interpretabilitycited by 38
  17. 2023Interpretable Machine Learning for Science with PySR and SymbolicRegression.jlcited by 36
  18. 2022Towards Data-and Knowledge-Driven Artificial Intelligence: A Survey on Neuro-Symbolic Computingcited by 34
  19. 2022Generalizing Goal-Conditioned Reinforcement Learning with Variational Causal Reasoningcited by 33
  20. 2023Discovering Causal Relations and Equations from Datacited by 33
  21. 2021AlphaEvolve: A Learning Framework to Discover Novel Alphas in Quantitative Investmentcited by 31
  22. 2022Thermodynamics-informed graph neural networkscited by 31
  23. 2022gLaSDI: Parametric Physics-informed Greedy Latent Space Dynamics Identificationcited by 30
  24. 2022Exhaustive Symbolic Regressioncited by 29
  25. 2023Logic-induced Diagnostic Reasoning for Semi-supervised Semantic Segmentationcited by 29
  26. 2021Noether Networks: Meta-Learning Useful Conserved Quantitiescited by 26
  27. 2023The transformative potential of machine learning for experiments in fluid mechanicscited by 26
  28. 2023Programmatically Grounded, Compositionally Generalizable Robotic Manipulationcited by 26
  29. 2023Discovering a reaction-diffusion model for Alzheimer's disease by combining PINNs with symbolic regressioncited by 25
  30. 2021An Extensible Benchmark Suite for Learning to Simulate Physical Systemscited by 24
  31. 2022Discovering Nonlinear PDEs from Scarce Data with Physics-encoded Learningcited by 24
  32. 2022Unravelling the Performance of Physics-informed Graph Neural Networks for Dynamical Systemscited by 23
  33. 2023Interpretable Prototype-based Graph Information Bottleneckcited by 23
  34. 2022Deep Learning and Symbolic Regression for Discovering Parametric Equationscited by 22
  35. 2022Learning Articulated Rigid Body Dynamics with Lagrangian Graph Neural Networkcited by 22
  36. 2023Deep Learning Symmetries and Their Lie Groups, Algebras, and Subalgebras from First Principlescited by 22
  37. 2021An Approach to Symbolic Regression Using Feyncited by 20
  38. 2022Multivariate Time Series Forecasting with Latent Graph Inferencecited by 20

50 most cited of 335 All 335 (JSON)