# Random Rule Forest (RRF): Interpretable and Manageable Ensembles of LLM-Generated Questions for Predicting Success from Unstructured Data

arXiv: https://arxiv.org/abs/2505.24622 · HTML: https://mlsanity.com/paper/2505.24622/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 15 references that are in the map.
- First posted on arXiv 30 May 2025.

## Builds on (most cited first, 10 of 10)

- [Wisdom of the Silicon Crowd: LLM Ensemble Prediction Capabilities Rival Human Crowd Accuracy](https://mlsanity.com/paper/2402.19379/) (2024)
- [HINT: Hierarchical Interaction Network for Trial Outcome Prediction Leveraging Web Data](https://mlsanity.com/paper/2102.04252/) (2021)
- [Tree Prompting: Efficient Task Adaptation without Fine-Tuning](https://mlsanity.com/paper/2310.14034/) (2023)
- [A Fused Large Language Model for Predicting Startup Success](https://mlsanity.com/paper/2409.03668/) (2024)
- [LLM-based feature generation from text for interpretable machine learning](https://mlsanity.com/paper/2409.07132/) (2024)
- [GPTree: Towards Explainable Decision-Making via LLM-powered Decision Trees](https://mlsanity.com/paper/2411.08257/) (2024)
- [Finding the unicorn: Predicting early stage startup success through a hybrid intelligence method](https://mlsanity.com/paper/2105.03360/) (2021)
- [Founder-GPT: Self-play to evaluate the Founder-Idea fit](https://mlsanity.com/paper/2312.12037/) (2023)
- [Policy Induction: Predicting Startup Success via Explainable Memory-Augmented In-Context Learning](https://mlsanity.com/paper/2505.21427/) (2025)
- [Reasoning-Based AI for Startup Evaluation (R.A.I.S.E.): A Memory-Augmented, Multi-Step Decision Framework](https://mlsanity.com/paper/2504.12090/) (2025)

## Built on by (most cited first, 1 of 1)

- [Metrics That Write Themselves: Evolving an Evaluator from Its Own Blind Spots](https://mlsanity.com/paper/2608.18744/) (2026)

## Abstract (arXiv)

Many high-stakes screening tasks require predicting rare outcomes from unstructured text, where errors are costly and decisions must be auditable. We introduce Random Rule Forest (RRF), an interpretable ensemble that uses a large language model (LLM) not as an end-to-end predictor but as a generator of simple YES/NO questions. Each question acts as a weak learner, and their responses are combined by a plain unit-weight vote into an auditable ``green-flags'' scorecard: enough independent positive signals indicate a higher chance of success. We argue this deliberate simplicity is a robust default when positives are scarce and learned weights are hard to estimate. We evaluate RRF in two low-base-rate domains. On early-stage startup screening from founder profiles, RRF produces a transparent scorecard whose precision is several times the base rate (with light expert input raising it further) and, unlike direct prompting, its operating point can be controlled directly. On an established Phase~I clinical-trial benchmark, RRF outperforms published baselines on the threshold-independent metrics PR-AUC and ROC-AUC. Together these show that LLMs can serve as auditable feature generators for high-stakes text-based decisions, combining transparency with competitive predictive performance.