mlsanityData to 7 Oct 2026

Deep Symbolic Regression for Recurrent Sequences

arXiv 2201.04600 ↗First posted 12 Jan 2022cs.LGcited by 53

Not on the map. Why?

Abstract (arXiv)

Symbolic regression, i.e. predicting a function from the observation of its values, is well-known to be a challenging task. In this paper, we train Transformers to infer the function or recurrence relation underlying sequences of integers or floats, a typical task in human IQ tests which has hardly been tackled in the machine learning literature. We evaluate our integer model on a subset of OEIS sequences, and show that it outperforms built-in Mathematica functions for recurrence prediction. We also demonstrate that our float model is able to yield informative approximations of out-of-vocabulary functions and constants, e.g. operatornamebessel0(x)≈ fracsin(x)+cos(x)√π x and 1.644934≈ π2/6. An interactive demonstration of our models is provided at https://symbolicregression.metademolab.com.

Key facts. Cited by 53 papers in the map (data through 7 Oct 2026). Builds on 16 of its 33 references that are in the map. First posted on arXiv 12 Jan 2022.

Builds on

16 of 33 references.

References · 16

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2019Analysing Mathematical Reasoning Abilities of Neural Modelscited by 331
  4. 2019Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradientscited by 294
  5. 2019Deep Learning for Symbolic Mathematicscited by 266
  6. 2021Neural Symbolic Regression that Scalescited by 185
  7. 2018Learning Equations for Extrapolation and Controlcited by 151
  8. 2015Neural GPUs Learn Algorithmscited by 118
  9. 2021SymbolicGPT: A Generative Transformer Model for Symbolic Regressioncited by 101
  10. 2019Integration of Neural Network-Based Symbolic Regression in Deep Learning for Scientific Discoverycited by 93
  11. 2018Neural Arithmetic Logic Unitscited by 69
  12. 2021Linear algebra with transformerscited by 62
4 more
  1. 2016Learning Continuous Semantic Representations of Symbolic Expressionscited by 25
  2. 2018Number Sequence Prediction Problems for Evaluating Computational Powers of Neural Networkscited by 6
  3. 2018Can machine learning identify interesting mathematics? An exploration using empirically observed lawscited by 4
  4. 2021Learning Mathematical Properties of Integerscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 13; 2023: 11; 2024: 16; 2025: 9; 2026: 4.

53 papers.

Sort:

By area:

  1. 2022End-to-end symbolic regression with transformerscited by 195
  2. 2022HyperTree Proof Search for Neural Theorem Provingcited by 192
  3. 2022Tutorial on amortized optimizationcited by 83
  4. 2022Symbolic Regression is NP-hardcited by 68
  5. 2023The No Free Lunch Theorem, Kolmogorov Complexity, and the Role of Inductive Biases in Machine Learningcited by 63
  6. 2022SymFormer: End-to-end symbolic regression using transformer-based architecturecited by 56
  7. 2024Towards a Foundation Model for Partial Differential Equations: Multi-Operator Learning and Extrapolationcited by 45
  8. 2023Interpretable Machine Learning for Science with PySR and SymbolicRegression.jlcited by 36
  9. 2024OmniPred: Language Models as Universal Regressorscited by 34
  10. 2024PROSE-FD: A Multimodal PDE Foundation Model for Learning Multiple Operators for Forecasting Fluid Dynamicscited by 33
  11. 2022SALSA: Attacking Lattice Cryptography with Transformerscited by 28
  12. 2023Predicting Ordinary Differential Equations with Transformerscited by 26
38 more
  1. 2023Deep Learning Symmetries and Their Lie Groups, Algebras, and Subalgebras from First Principlescited by 22
  2. 2023Controllable Neural Symbolic Regressioncited by 19
  3. 2025Position: The Future of Bayesian Prediction Is Prior-Fittedcited by 19
  4. 2024Transforming the Bootstrap: Using Transformers to Compute Scattering Amplitudes in Planar N = 4 Super Yang-Mills Theorycited by 17
  5. 2023PROSE: Predicting Operators and Symbolic Expressions using Multimodal Transformerscited by 16
  6. 2024Symbolic regression via MDLformer-guided search: from minimizing prediction error to minimizing description lengthcited by 16
  7. 2023Toward Physically Plausible Data-Driven Models: A Novel Neural Network Approach to Symbolic Regressioncited by 14
  8. 2023Efficient Generator of Mathematical Expressions for Symbolic Regressioncited by 13
  9. 2024LeMON: Learning to Learn Multi-Operator Networkscited by 13
  10. 2022SYMBA: Symbolic Computation of Squared Amplitudes in High Energy Physics with Machine Learningcited by 12
  11. 2022Machine Learning and Deep Learning -- A review for Ecologistscited by 11
  12. 2025Decoding-based Regressioncited by 11
  13. 2023Priors for symbolic regressioncited by 10
  14. 2024Neuro-Symbolic AI: Explainability, Challenges, and Future Trendscited by 10
  15. 2023Alien Codingcited by 9
  16. 2022Learning Program Synthesis for Integer Sequences from Scratchcited by 8
  17. 2022Is the Machine Smarter than the Theorist: Deriving Formulas for Particle Kinematics with Symbolic Regressioncited by 8
  18. 2024Can Transformers Do Enumerative Geometry?cited by 8
  19. 2024In Context Learning and Reasoning for Symbolic Regression with Large Language Modelscited by 7
  20. 2022FACT: Learning Governing Abstractions Behind Integer Sequencescited by 6
  21. 2025PI-MFM: Physics-informed multimodal foundation model for solving partial differential equationscited by 6
  22. 2023Oracle-Preserving Latent Flowscited by 5
  23. 2024A Comparison of Recent Algorithms for Symbolic Regression to Genetic Programmingcited by 5
  24. 2024Learning the Simplicity of Scattering Amplitudescited by 5
  25. 2025Grammar-based Ordinary Differential Equation Discoverycited by 5
  26. 2024Symbolic Equation Solving via Reinforcement Learningcited by 4
  27. 2022Periodic Extrapolative Generalisation in Neural Networkscited by 3
  28. 2024Learning Elementary Cellular Automata with Transformerscited by 3
  29. 2026Neuro-Symbolic ODE Discovery with Latent Grammar Flowcited by 2
  30. 2025Int2Int: a framework for mathematics with transformerscited by 1
  31. 2025Graph-Eq: Discovering Mathematical Equations using Graph Generative Modelscited by 1
  32. 2025Neuro-Evolutionary Approach to Physics-Aware Symbolic Regressioncited by 1
  33. 2025Rule-Based Moral Principles for Explaining Uncertainty in Natural Language Generationcited by 1
  34. 2025Symbolic-Diffusion: Deep Learning Based Symbolic Regression with D3PM Discrete Token Diffusioncited by 1
  35. 2026From Data to Theory: Autonomous Large Language Model Agents for Materials Sciencecited by 1
  36. 2022KitBit: A New AI Model for Solving Intelligence Tests and Numerical Seriescited by 0
  37. 2024Automated Discovery of Integral with Deep Learningcited by 0
  38. 2024SPINEX_ Symbolic Regression: Similarity-based Symbolic Regression with Explainable Neighbors Explorationcited by 0

50 most cited of 53 All 53 (JSON)