SCP-NL2TL: Selective Conformal Prediction with Semantic Verification for Natural Language to Temporal Logic Specifications
Not on the map. Why?
Abstract (arXiv)
Translating natural language instructions into machine-interpretable formal specifications enables robots and autonomous systems to plan, reason, and formally verify their behavior. However, existing translation models typically generate a specification for every input, even when the result is unreliable or fails to capture the user's intent, creating risks in safety-critical applications. Inspired by selective conformal prediction, we propose a selective translation framework that not only generates formal specifications but also determines when they can be trusted. Reliability is scored by two complementary black-box signals, the fidelity of the specification back-translated into natural language and the dispersion of repeated translations under exact semantic equivalence, which fail on different errors and jointly separate incorrect translations more sharply than either alone. Conformal risk control calibrates this score into a decision that accepts a specification or abstains, with a distribution-free bound on the rate at which incorrect specifications are accepted for execution, and a conformal anomaly detector on instruction embeddings screens out-of-distribution inputs before any translation is attempted. The proposed framework is general across formal specification languages, with experiments on Signal Temporal Logic (STL), Linear Temporal Logic (LTL), and geometric Spatio-Temporal Logic (SpaTiaL) demonstrating improved translation reliability, robustness under the evaluated cross-tier shifts, and effective uncertainty-aware abstention. This work establishes a foundation for trustworthy natural language interfaces by enabling AI systems to recognize when generated specifications may not be reliable.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 45 of its 61 references that are in the map. First posted on arXiv 5 Aug 2026.
Builds on
45 of 61 references.
References · 45
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
- 2022Out-of-Distribution Detection with Deep Nearest Neighborscited by 600
- 2019Conformal Prediction Under Covariate Shiftcited by 519
- 2022Conformal prediction beyond exchangeabilitycited by 355
- 2019SelectiveNet: A Deep Neural Network with an Integrated Reject Optioncited by 350
- 2023Robots That Ask For Help: Uncertainty Alignment for Large Language Model Plannerscited by 308
- 2022Conformal Risk Controlcited by 299
- 2023Conformal Language Modelingcited by 170
- 2023Shifting Attention to Relevance: Towards the Predictive Uncertainty Quantification of Free-Form Large Language Modelscited by 150
- 2021Testing for Outliers with Conformal p-valuescited by 134
- 2024Language Models with Conformal Factuality Guaranteescited by 131
33 more
- 2020Semantic Evaluation for Text-to-SQL with Distilled Test Suitescited by 126
- 2022Safe Planning in Dynamic Environments using Conformal Predictioncited by 119
- 2024Large language model validity via enhanced conformal prediction methodscited by 92
- 2022Selection by Prediction with Conformal p-valuescited by 69
- 2024API Is Enough: Conformal Prediction for Large Language Models Without Logit-Accesscited by 69
- 2024ConU: Conformal Uncertainty in Large Language Models with Correctness Coverage Guaranteescited by 65
- 2023nl2spec: Interactively Translating Unstructured Natural Language to Temporal Logics with Large Language Modelscited by 60
- 2023NL2TL: Transforming Natural Languages to Temporal Logics using Large Language Modelscited by 59
- 2020Backpropagation through Signal Temporal Logic Specifications: Infusing Logical Structure into Gradient-Based Methodscited by 54
- 2022Calibrated Interpretation: Confidence Estimation in Semantic Parsingcited by 33
- 2023Non-Exchangeable Conformal Risk Controlcited by 29
- 2020Revisiting Round-Trip Translation for Quality Estimationcited by 24
- 2025SConU: Selective Conformal Uncertainty in Large Language Modelscited by 23
- 2020Learning a natural-language to LTL executable semantic parser for grounded roboticscited by 22
- 2021DeepSTL -- From English Requirements to Signal Temporal Logiccited by 22
- 2023Conformal Temporal Logic Planning using Large Language Modelscited by 21
- 2024Probabilistically Correct Language-based Multi-Robot Planning using Conformal Predictioncited by 18
- 2023Prompt Risk Control: A Rigorous Framework for Responsible Deployment of Large Language Modelscited by 17
- 2023Selective Generation for Controllable Language Modelscited by 15
- 2018Probabilistic Model Checking of Robots Deployed in Extreme Environmentscited by 12
- 2022Learning Interpretable Temporal Properties from Positive Examples Onlycited by 12
- 2023A Unified View of Evaluation Metrics for Structured Predictioncited by 11
- 2025Confidence Estimation for Error Detection in Text-to-SQL Systemscited by 11
- 2025ConformalNL2LTL: Translating Natural Language Instructions into Temporal Logic Formulas with Conformal Correctness Guaranteescited by 11
- 2025Selective Conformal Risk Controlcited by 11
- 2022Learning Signal Temporal Logic through Neural Network for Interpretable Classificationcited by 10
- 2024TLINet: Differentiable Neural Network Temporal Logic Inferencecited by 9
- 2025Enhancing Transformation from Natural Language to Signal Temporal Logic Using LLMs with Diverse External Knowledgecited by 9
- 2025Grammar-Forced Translation of Natural Language to Temporal Logic using LLMscited by 8
- 2025Conformal Prediction for Signal Temporal Logic Inferencecited by 3
- 2025NL2SpaTiaL: Generating Geometric Spatio-Temporal Logic Specifications from Natural Language for Manipulation Taskscited by 3
- 2026Conformalized Signal Temporal Logic Inference under Covariate Shiftcited by 1
- 2026Differentiable SpaTiaL: Symbolic Learning and Reasoning with Geometric Temporal Logic for Manipulation Taskscited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.