# Tree-Based Deep Learning for Ranking Symbolic Integration Algorithms

arXiv: https://arxiv.org/abs/2508.06383 · HTML: https://mlsanity.com/paper/2508.06383/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 40 references that are in the map.
- First posted on arXiv 8 Aug 2025.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks](https://mlsanity.com/paper/1503.00075/) (2015)
- [Deep Learning for Symbolic Mathematics](https://mlsanity.com/paper/1912.01412/) (2019)
- [End-to-end symbolic regression with transformers](https://mlsanity.com/paper/2204.10532/) (2022)
- [Tree Transformer: Integrating Tree Structures into Self-Attention](https://mlsanity.com/paper/1909.06639/) (2019)
- [ODEFormer: Symbolic Regression of Dynamical Systems with Transformers](https://mlsanity.com/paper/2310.05573/) (2023)
- [Symbolic Brittleness in Sequence Models: on Systematic Generalization in Symbolic Mathematics](https://mlsanity.com/paper/2109.13986/) (2021)
- [Can Neural Networks Learn Symbolic Rewriting?](https://mlsanity.com/paper/1911.04873/) (2019)
- [The Use of Deep Learning for Symbolic Integration: A Review of (Lample and Charton, 2019)](https://mlsanity.com/paper/1912.05752/) (2019)
- [Algebraic Positional Encodings](https://mlsanity.com/paper/2312.16045/) (2023)
- [Explainable AI Insights for Symbolic Computation: A case study on selecting the variable ordering for cylindrical algebraic decomposition](https://mlsanity.com/paper/2304.12154/) (2023)
- [Learning to Compute Gr"obner Bases](https://mlsanity.com/paper/2311.12904/) (2023)
- [Learning Guided Automated Reasoning: A Brief Survey](https://mlsanity.com/paper/2403.04017/) (2024)
- [Using Machine Learning to Improve Cylindrical Algebraic Decomposition](https://mlsanity.com/paper/1804.10520/) (2018)
- [Generating Elementary Integrable Expressions](https://mlsanity.com/paper/2306.15572/) (2023)
- [Symbolic Integration Algorithm Selection with Machine Learning: LSTMs vs Tree LSTMs](https://mlsanity.com/paper/2404.14973/) (2024)
- [Lessons on Datasets and Paradigms in Machine Learning for Symbolic Computation: A Case Study on CAD](https://mlsanity.com/paper/2401.13343/) (2024)
- [The Liouville Generator for Producing Integrable Expressions](https://mlsanity.com/paper/2406.11631/) (2024)
- [Transformers to Predict the Applicability of Symbolic Integration Routines](https://mlsanity.com/paper/2410.23948/) (2024)

## Built on by (most cited first, 1 of 1)

- [Enhancing Transformer Representations of Symbolic ODE Expressions](https://mlsanity.com/paper/2609.24746/) (2026)

## Abstract (arXiv)

Symbolic indefinite integration in Computer Algebra Systems such as Maple involves selecting the most effective algorithm from multiple available methods. Not all methods will succeed for a given problem, and when several do, the results, though mathematically equivalent, can differ greatly in presentation complexity. Traditionally, this choice has been made with minimal consideration of the problem instance, leading to inefficiencies. We present a machine learning (ML) approach using tree-based deep learning models within a two-stage architecture: first identifying applicable methods for a given instance, then ranking them by predicted output complexity. Furthermore, we find representing mathematical expressions as tree structures significantly improves performance over sequence-based representations, and our two-stage framework outperforms alternative ML formulations. Using a diverse dataset generated by six distinct data generators, our models achieve nearly 90% accuracy in selecting the optimal method on a 70,000 example holdout test set. On an independent out-of-distribution benchmark from Maple's internal test suite, our tree transformer model maintains strong generalisation, outperforming Maple's built-in selector and prior ML approaches. These results highlight the critical role of data representation and problem framing in ML for symbolic computation, and we expect our methodology to generalise effectively to similar optimisation problems in mathematical software.