# Learning Formal Specifications from Membership and Preference Queries

arXiv: https://arxiv.org/abs/2307.10434 · HTML: https://mlsanity.com/paper/2307.10434/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 37 references that are in the map.
- First posted on arXiv 19 Jul 2023.

## Builds on (most cited first, 11 of 11)

- [Cooperative Inverse Reinforcement Learning](https://mlsanity.com/paper/1606.03137/) (2016)
- [Batch Active Preference-Based Learning of Reward Functions](https://mlsanity.com/paper/1810.04303/) (2018)
- [Learning Reward Functions by Integrating Human Demonstrations and Preferences](https://mlsanity.com/paper/1906.08928/) (2019)
- [Data-Efficient Learning of Natural Language to Linear Temporal Logic Translators for Robot Task Specification](https://mlsanity.com/paper/2303.08006/) (2023)
- [Risk-Aware Active Inverse Reinforcement Learning](https://mlsanity.com/paper/1901.02161/) (2019)
- [Explaining Multi-stage Tasks by Learning Temporal Logic Formulas from Suboptimal Demonstrations](https://mlsanity.com/paper/2006.02411/) (2020)
- [A Corroborative Approach to Verification and Validation of Human--Robot Teams](https://mlsanity.com/paper/1608.07403/) (2016)
- [Active Learning of Points-To Specifications](https://mlsanity.com/paper/1711.03239/) (2017)
- [Thresholding Bandit Problem with Both Duels and Pulls](https://mlsanity.com/paper/1910.06368/) (2019)
- [Noise-Tolerant Interactive Learning from Pairwise Comparisons](https://mlsanity.com/paper/1704.05820/) (2017)
- [Joint Learning of Policy with Unknown Temporal Constraints for Safe Reinforcement Learning](https://mlsanity.com/paper/2305.00576/) (2023)

## Built on by (most cited first, 3 of 3)

- [Event-B Agent: Towards LLM Agent for Formal Model Synthesis and Repair](https://mlsanity.com/paper/2605.17475/) (2026)
- [Learning Quantitative Automata Modulo Theories](https://mlsanity.com/paper/2411.10601/) (2024)
- [Automata Learning of Preferences over Temporal Logic Formulas from Pairwise Comparisons](https://mlsanity.com/paper/2505.18030/) (2025)

## Abstract (arXiv)

Active learning is a well-studied approach to learning formal specifications, such as automata. In this work, we extend active specification learning by proposing a novel framework that strategically requests a combination of membership labels and pair-wise preferences, a popular alternative to membership labels. The combination of pair-wise preferences and membership labels allows for a more flexible approach to active specification learning, which previously relied on membership labels only. We instantiate our framework in two different domains, demonstrating the generality of our approach. Our results suggest that learning from both modalities allows us to robustly and conveniently identify specifications via membership and preferences.