# RLIE: Rule Generation with Logistic Regression, Iterative Refinement, and Evaluation for Large Language Models

arXiv: https://arxiv.org/abs/2510.19698 · HTML: https://mlsanity.com/paper/2510.19698/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 35 references that are in the map.
- First posted on arXiv 22 Oct 2025.

## Builds on (most cited first, 18 of 18)

- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Large Language Models for Automated Open-domain Scientific Hypotheses Discovery](https://mlsanity.com/paper/2309.02726/) (2023)
- [Phenomenal Yet Puzzling: Testing Inductive Reasoning Capabilities of Language Models with Hypothesis Refinement](https://mlsanity.com/paper/2310.08559/) (2023)
- [Learning programs by learning from failures](https://mlsanity.com/paper/2005.02259/) (2020)
- [MOOSE-Chem: Large Language Models for Rediscovering Unseen Chemistry Scientific Hypotheses](https://mlsanity.com/paper/2410.07076/) (2024)
- [Hypothesis Generation with Large Language Models](https://mlsanity.com/paper/2404.04326/) (2024)
- [Scalable Bayesian Rule Lists](https://mlsanity.com/paper/1602.08610/) (2016)
- [Inductive logic programming at 30](https://mlsanity.com/paper/2102.10556/) (2021)
- [HypoBench: Towards Systematic and Principled Benchmarking for Hypothesis Generation](https://mlsanity.com/paper/2504.11524/) (2025)
- [Learning Accurate and Interpretable Decision Rule Sets from Neural Networks](https://mlsanity.com/paper/2103.02826/) (2021)
- [Neuro-Symbolic Hierarchical Rule Induction](https://mlsanity.com/paper/2112.13418/) (2021)
- [Human-like Few-Shot Learning via Bayesian Reasoning over Natural Language](https://mlsanity.com/paper/2306.02797/) (2023)
- [Explaining Datasets in Words: Statistical Models with Natural Language Parameters](https://mlsanity.com/paper/2409.08466/) (2024)
- [Learning MDL logic programs from noisy data](https://mlsanity.com/paper/2308.09393/) (2023)
- [Truly Unordered Probabilistic Rule Sets for Multi-class Classification](https://mlsanity.com/paper/2206.08804/) (2022)
- [Generalisation Through Negation and Predicate Invention](https://mlsanity.com/paper/2301.07629/) (2023)
- [RuAG: Learned-rule-augmented Generation for Large Language Models](https://mlsanity.com/paper/2411.03349/) (2024)
- [Neuro-Symbolic Rule Lists](https://mlsanity.com/paper/2411.06428/) (2024)

## Abstract (arXiv)

Large Language Models (LLMs) can propose rules in natural language, sidestepping the need for a predefined predicate space in traditional rule learning. Yet many LLM-based approaches ignore interactions among rules, and the opportunity to couple LLMs with probabilistic rule learning for robust inference remains underexplored. We present RLIE, a unified framework that integrates LLMs with probabilistic modeling to learn a set of weighted rules. RLIE has four stages: (1) Rule generation, where an LLM proposes and filters candidates; (2) Logistic regression, which learns probabilistic weights for global selection and calibration; (3) Iterative refinement, which updates the rule set using prediction errors; and (4) Evaluation, which compares the weighted rule set as a direct classifier with methods that inject rules into an LLM. We evaluate multiple inference strategies on real-world datasets. Applying rules directly with their learned weights yields superior performance, whereas prompting LLMs with the rules, weights, and logistic-model outputs surprisingly degrades accuracy. This supports the view that LLMs excel at semantic generation and interpretation but are less reliable for precise probabilistic integration. RLIE clarifies the potential and limitations of LLMs for inductive reasoning and couples them with classic probabilistic rule combination methods to enable more reliable neuro-symbolic reasoning.