# Bridging the Domain Gap in Equation Distillation with Reinforcement Feedback

arXiv: https://arxiv.org/abs/2505.15572 · HTML: https://mlsanity.com/paper/2505.15572/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 62 references that are in the map.
- First posted on arXiv 21 May 2025.

## Builds on (most cited first, 30 of 34)

- [AI Feynman: a Physics-Inspired Method for Symbolic Regression](https://mlsanity.com/paper/1905.11481/) (2019)
- [Discovering Symbolic Models from Deep Learning with Inductive Biases](https://mlsanity.com/paper/2006.11287/) (2020)
- [Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients](https://mlsanity.com/paper/1912.04871/) (2019)
- [Contemporary Symbolic Regression Methods and their Relative Performance](https://mlsanity.com/paper/2107.14351/) (2021)
- [End-to-end symbolic regression with transformers](https://mlsanity.com/paper/2204.10532/) (2022)
- [Neural Symbolic Regression that Scales](https://mlsanity.com/paper/2106.06427/) (2021)
- [AI Feynman 2.0: Pareto-optimal symbolic regression exploiting graph modularity](https://mlsanity.com/paper/2006.10782/) (2020)
- [PMLB: A Large Benchmark Suite for Machine Learning Evaluation and Comparison](https://mlsanity.com/paper/1703.00512/) (2017)
- [SymbolicGPT: A Generative Transformer Model for Symbolic Regression](https://mlsanity.com/paper/2106.14131/) (2021)
- [Symbolic Regression via Neural-Guided Genetic Programming Population Seeding](https://mlsanity.com/paper/2111.00053/) (2021)
- [Integration of Neural Network-Based Symbolic Regression in Deep Learning for Scientific Discovery](https://mlsanity.com/paper/1912.04825/) (2019)
- [Transformer-based Planning for Symbolic Regression](https://mlsanity.com/paper/2303.06833/) (2023)
- [Bayesian Symbolic Regression](https://mlsanity.com/paper/1910.08892/) (2019)
- [Symbolic Physics Learner: Discovering governing equations via Monte Carlo tree search](https://mlsanity.com/paper/2205.13134/) (2022)
- [Learning concise representations for regression by evolving networks of trees](https://mlsanity.com/paper/1807.00981/) (2018)
- [Epsilon-Lexicase Selection for Regression](https://mlsanity.com/paper/1905.13266/) (2019)
- [Self-optimizing Feature Generation via Categorical Hashing Representation and Hierarchical Reinforcement Crossing](https://mlsanity.com/paper/2309.04612/) (2023)
- [Evolutionary Large Language Model for Automated Feature Transformation](https://mlsanity.com/paper/2405.16203/) (2024)
- [Deep Generative Symbolic Regression](https://mlsanity.com/paper/2401.00282/) (2023)
- [LLM-Enhanced User-Item Interactions: Leveraging Edge Information for Optimized Recommendations](https://mlsanity.com/paper/2402.09617/) (2024)
- [Deep Learning and Symbolic Regression for Discovering Parametric Equations](https://mlsanity.com/paper/2207.00529/) (2022)
- [Feature Selection as Deep Sequential Generative Learning](https://mlsanity.com/paper/2403.03838/) (2024)
- [Unsupervised Generative Feature Transformation via Graph Contrastive Pre-training and Multi-objective Fine-tuning](https://mlsanity.com/paper/2405.16879/) (2024)
- [Double Correction Framework for Denoising Recommendation](https://mlsanity.com/paper/2405.11272/) (2024)
- [CT-PatchTST: Channel-Time Patch Time-Series Transformer for Long-Term Renewable Energy Forecasting](https://mlsanity.com/paper/2501.08620/) (2025)
- [Knockoff-Guided Feature Selection via A Single Pre-trained Reinforced Agent](https://mlsanity.com/paper/2403.04015/) (2024)
- [Neuro-Symbolic Embedding for Short and Effective Feature Selection via Autoregressive Generation](https://mlsanity.com/paper/2404.17157/) (2024)
- [A Survey on Data-Centric AI: Tabular Learning from Reinforcement Learning and Generative AI Perspective](https://mlsanity.com/paper/2502.08828/) (2025)
- [Multimodality Invariant Learning for Multimedia-Based New Item Recommendation](https://mlsanity.com/paper/2405.15783/) (2024)
- [Reinforcement Feature Transformation for Polymer Property Performance Prediction](https://mlsanity.com/paper/2409.15616/) (2024)

## Built on by (most cited first, 4 of 4)

- [Efficient Post-Training Refinement of Latent Reasoning in Large Language Models](https://mlsanity.com/paper/2506.08552/) (2025)
- [Teaching Time Series to See and Speak: Forecasting with Aligned Visual and Textual Perspectives](https://mlsanity.com/paper/2506.24124/) (2025)
- [LLM-ML Teaming: Integrated Symbolic Decoding and Gradient Search for Valid and Stable Generative Feature Transformation](https://mlsanity.com/paper/2506.09085/) (2025)
- [Data-Efficient Symbolic Regression via Foundation Model Distillation](https://mlsanity.com/paper/2508.19487/) (2025)

## Abstract (arXiv)

The data-to-equation (Data2Eqn) task aims to discover interpretable mathematical equations that map observed values to labels, offering physical insights and broad applicability across academic and industrial domains. Genetic programming and traditional deep learning-based approaches suffer from search inefficiency and poor generalization on small task-specific datasets. Foundation models showed promise in this area, but existing approaches suffer from: 1) They are pretrained on general-purpose data distributions, making them less effective for domain-specific tasks; and 2) their training objectives focus on token-level alignment, overlooking mathematical semantics, which can lead to inaccurate equations. To address these issues, we aim to enhance the domain adaptability of foundation models for Data2Eqn tasks. In this work, we propose a reinforcement learning-based finetuning framework that directly optimizes the generation policy of a pretrained model through reward signals derived from downstream numerical fitness. Our method allows the model to adapt to specific and complex data distributions and generate mathematically meaningful equations. Extensive experiments demonstrate that our approach improves both the accuracy and robustness of equation generation under complex distributions.