# Assessing the Interpretability of Programmatic Policies with Large Language Models

arXiv: https://arxiv.org/abs/2311.06979 · HTML: https://mlsanity.com/paper/2311.06979/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 26 references that are in the map.
- First posted on arXiv 12 Nov 2023.

## Builds on (most cited first, 8 of 8)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Interpretable Explanations of Black Boxes by Meaningful Perturbation](https://mlsanity.com/paper/1704.03296/) (2017)
- [Programmatically Interpretable Reinforcement Learning](https://mlsanity.com/paper/1804.02477/) (2018)
- [Verifiable Reinforcement Learning via Policy Extraction](https://mlsanity.com/paper/1805.08328/) (2018)
- [What can we Learn Even From the Weakest? Learning Sketches for Programmatic Strategies](https://mlsanity.com/paper/2203.11912/) (2022)
- [Evaluating Code Readability and Legibility: An Examination of Human-centric Studies](https://mlsanity.com/paper/2110.00785/) (2021)
- [Choosing Well Your Opponents: How to Guide the Synthesis of Programmatic Strategies](https://mlsanity.com/paper/2307.04893/) (2023)

## Built on by (most cited first, 3 of 3)

- [Interpretable and Editable Programmatic Tree Policies for Reinforcement Learning](https://mlsanity.com/paper/2405.14956/) (2024)
- [Evaluating Interpretable Reinforcement Learning by Distilling Policies into Programs](https://mlsanity.com/paper/2503.08322/) (2025)
- [DiPRL: Learning Discrete Programmatic Policies via Architecture Entropy Regularization](https://mlsanity.com/paper/2605.18508/) (2026)

## Abstract (arXiv)

Although the synthesis of programs encoding policies often carries the promise of interpretability, systematic evaluations were never performed to assess the interpretability of these policies, likely because of the complexity of such an evaluation. In this paper, we introduce a novel metric that uses large-language models (LLM) to assess the interpretability of programmatic policies. For our metric, an LLM is given both a program and a description of its associated programming language. The LLM then formulates a natural language explanation of the program. This explanation is subsequently fed into a second LLM, which tries to reconstruct the program from the natural-language explanation. Our metric then measures the behavioral similarity between the reconstructed program and the original. We validate our approach with synthesized and human-crafted programmatic policies for playing a real-time strategy game, comparing the interpretability scores of these programmatic policies to obfuscated versions of the same programs. Our LLM-based interpretability score consistently ranks less interpretable programs lower and more interpretable ones higher. These findings suggest that our metric could serve as a reliable and inexpensive tool for evaluating the interpretability of programmatic policies.