# CDRL: Certification-Driven Reinforcement Learning for Neutrino Flavor Model Discovery

arXiv: https://arxiv.org/abs/2608.20686 · HTML: https://mlsanity.com/paper/2608.20686/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning.
- Builds on 18 of its 70 references that are in the map.
- First posted on arXiv 21 Aug 2026.

## Placed in

- LLM reasoning / [Reinforcement learning for reasoning](https://mlsanity.com/lr/reinforcement-learning-for-reasoning/)

## Builds on (most cited first, 18 of 18)

- [OpenAI GPT-5 System Card](https://mlsanity.com/paper/2601.03267/) (2025)
- [Learning to Plan Chemical Syntheses](https://mlsanity.com/paper/1708.04202/) (2017)
- [LLMs Can't Plan, But Can Help Planning in LLM-Modulo Frameworks](https://mlsanity.com/paper/2402.01817/) (2024)
- [Verifiable Reinforcement Learning via Policy Extraction](https://mlsanity.com/paper/1805.08328/) (2018)
- [A Survey on Deep Learning for Theorem Proving](https://mlsanity.com/paper/2404.09939/) (2024)
- [Machine Learning in the Search for New Fundamental Physics](https://mlsanity.com/paper/2112.03769/) (2021)
- [Active learning machine learns to create new quantum experiments](https://mlsanity.com/paper/1706.00868/) (2017)
- [Neuro-Symbolic AI in 2024: A Systematic Review](https://mlsanity.com/paper/2501.05435/) (2025)
- [Quantum Circuit Optimization with AlphaTensor](https://mlsanity.com/paper/2402.14396/) (2024)
- [Neurosymbolic Reinforcement Learning and Planning: A Survey](https://mlsanity.com/paper/2309.01038/) (2023)
- [CoTran: An LLM-based Code Translator using Reinforcement Learning with Feedback from Compiler and Symbolic Execution](https://mlsanity.com/paper/2306.06755/) (2023)
- [A Theory of Formal Synthesis via Inductive Learning](https://mlsanity.com/paper/1505.03953/) (2015)
- [RLSF: Fine-tuning LLMs via Symbolic Feedback](https://mlsanity.com/paper/2405.16661/) (2024)
- [Exploring the flavor structure of quarks and leptons with reinforcement learning](https://mlsanity.com/paper/2304.14176/) (2023)
- [Reinforcement learning-based statistical search strategy for an axion model from flavor](https://mlsanity.com/paper/2409.10023/) (2024)
- [HordeSat: A Massively Parallel Portfolio SAT Solver](https://mlsanity.com/paper/1505.03340/) (2015)
- [Towards AI-assisted Neutrino Flavor Theory Design](https://mlsanity.com/paper/2506.08080/) (2025)
- [Satisfiability and Synthesis Modulo Oracles](https://mlsanity.com/paper/2107.13477/) (2021)

## Built on by (most cited first, 2 of 2)

- [Towards Certificate-Driven Software Porting: A Self-Improving Agentic Harness for Scientific Program Optimization](https://mlsanity.com/paper/2609.34069/) (2026)
- [Searching for BSM Experimental Signatures with Large Lagrangian Models](https://mlsanity.com/paper/2609.38309/) (2026)

## Abstract (arXiv)

Many scientific discovery problems require searching combinatorial hypothesis spaces under complex domain constraints. Reinforcement learning (RL) offers a promising approach, but existing methods rely on scalar rewards that provide limited information about why candidate solutions fail, leading agents to repeatedly explore invalid regions. We introduce Certification-Driven Reinforcement Learning (CDRL), a framework that leverages structured feedback from symbolic reasoning tools. When a candidate violates domain constraints, these tools produce certificates identifying the actions responsible for failure. CDRL converts these certificates into reusable constraints that eliminate classes of invalid solutions and guide exploration toward valid regions. We evaluate CDRL on neutrino flavor model discovery in theoretical particle physics, where the hypothesis space exceeds 1026 possible models, and compare it with the state-of-the-art RL approach previously used for this task. Across three theory spaces, CDRL achieves up to 1.95× higher valid model rates and up to 6.33× higher neutrino model rates while evaluating up to 4× fewer candidates. We further extract 40 interpretable rules from search trajectories using a post-hoc decision-tree framework and show that reusing them as soft constraints yields gains of up to 2× in valid model rates and 3× in neutrino model discovery across all three theory spaces. These results suggest that CDRL uncovers reusable structure in combinatorial search spaces and provides a general framework for scientific model discovery.