mlsanityData to 14 Oct 2026

Faithful Autoformalization via Roundtrip Verification and Repair

arXiv 2604.25031 ↗First posted 27 Apr 2026cs.CL cs.AIcited by 8

Not on the map. Why?

Abstract (arXiv)

When an LLM formalizes natural language, how do we know the output is faithful? We propose a roundtrip verification approach which does not require ground-truth annotations: formalize a statement, translate the result back to natural language, re-formalize, and use a formal tool to check logical equivalence. When the two formalizations agree, this provides evidence of a faithful formalization. When they disagree, a stage-level diagnosis localizes the error to a specific translation step, and a scoped repair operator attempts to correct that step. We evaluate the framework on two statutory domains (the Texas Transportation Code and the Texas Parks and Wildlife Code) using two LLMs (Claude Opus~4.6 and GPT-5.2) with three repair baselines. Diagnosis-guided scoped repair is the most effective method, with effectiveness contingent on the reliability of the diagnosis function. Across both domains and both models, under our full repair system, rules that fail the equivalence check show 1.4x-2.5x more natural language inference (NLI) drift than rules that pass it.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 28 of its 52 references that are in the map. First posted on arXiv 27 Apr 2026.

Builds on

28 of 52 references.

References · 28

  1. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  2. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  3. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
  4. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,282
  5. 2023Teaching Large Language Models to Self-Debugcited by 988
  6. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 900
  7. 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 876
  8. 2023LeanDojo: Theorem Proving with Retrieval-Augmented Language Modelscited by 427
  9. 2021SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarizationcited by 407
  10. 2021MiniF2F: a cross-system benchmark for formal Olympiad-level mathematicscited by 391
  11. 2023Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoningcited by 391
  12. 2022Draft, Sketch, and Prove: Guiding Formal Theorem Provers with Informal Proofscited by 291
16 more
  1. 2022Autoformalization with Large Language Modelscited by 263
  2. 2023ProofNet: Autoformalizing and Formally Proving Undergraduate-Level Mathematicscited by 192
  3. 2023LINC: A Neurosymbolic Approach for Logical Reasoning by Combining Language Models with First-Order Logic Proverscited by 168
  4. 2024Lean Workbook: A large-scale Lean problem set formalized from natural language math problemscited by 122
  5. 2023SatLM: Satisfiability-Aided Language Models Using Declarative Promptingcited by 102
  6. 2023Can GPT-3 Perform Statutory Reasoning?cited by 73
  7. 2023Clover: Closed-Loop Verifiable Code Generationcited by 67
  8. 2020A Dataset for Statutory Reasoning in Tax Law Entailment and Question Answeringcited by 62
  9. 2024Herald: A Natural Language Annotated Lean 4 Datasetcited by 59
  10. 2024Process-Driven Autoformalization in Lean 4cited by 45
  11. 2024Autoformalize Mathematical Statements by Symbolic Equivalence and Semantic Consistencycited by 41
  12. 2020Revisiting Round-Trip Translation for Quality Estimationcited by 24
  13. 2024Unsupervised Evaluation of Code LLMs with Round-Trip Correctnesscited by 23
  14. 2025ReForm: Reflective Autoformalization with Prospective Bounded Sequence Optimizationcited by 20
  15. 2025ProofBridge: Auto-Formalization of Natural Language Proofs in Lean via Joint Embeddingscited by 18
  16. 2024TR2MTL: LLM based framework for Metric Temporal Logic Formalization of Traffic Rulescited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 8.

8 papers.