# Stratified Consistency Distillation for Natural Language Formalization

arXiv: https://arxiv.org/abs/2608.30258 · HTML: https://mlsanity.com/paper/2608.30258/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 17 of its 28 references that are in the map.
- First posted on arXiv 31 Aug 2026.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 17 of 17)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [Fine-Tuning Language Models from Human Preferences](https://mlsanity.com/paper/1909.08593/) (2019)
- [Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations](https://mlsanity.com/paper/2312.08935/) (2023)
- [MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models](https://mlsanity.com/paper/2309.12284/) (2023)
- [Complexity-Based Prompting for Multi-Step Reasoning](https://mlsanity.com/paper/2210.00720/) (2022)
- [Divide and Translate: Compositional First-Order Logic Translation and Verification for Complex Logical Reasoning](https://mlsanity.com/paper/2410.08047/) (2024)
- [Accelerating RL for LLM Reasoning with Optimal Advantage Regression](https://mlsanity.com/paper/2505.20686/) (2025)
- [Avoiding mathbfexp(Rmax) scaling in RLHF through Preference-based Exploration](https://mlsanity.com/paper/2502.00666/) (2025)
- [Scaling In-Context Online Learning Capability of LLMs via Cross-Episode Meta-RL](https://mlsanity.com/paper/2602.04089/) (2026)
- [ProTransformer: Robustify Transformers via Plug-and-Play Paradigm](https://mlsanity.com/paper/2410.23182/) (2024)
- [Modulated Diffusion: Accelerating Generative Modeling with Modulated Quantization](https://mlsanity.com/paper/2506.22463/) (2025)
- [Robustness Reprogramming for Representation Learning](https://mlsanity.com/paper/2410.04577/) (2024)
- [CENTILE: A Telemetry Foundation Model Evaluated by the Decisions It Drives](https://mlsanity.com/paper/2608.01725/) (2026)

## Built on by (most cited first, 1 of 1)

- [MInTRL: Off-policy Intervention can boost On-policy RL](https://mlsanity.com/paper/2609.12419/) (2026)

## Abstract (arXiv)

Neurosymbolic reasoning has shown promising success in addressing complex reasoning tasks by combining large language models (LLMs) and symbolic solvers. While this approach shows promise, a fundamental challenge remains: improving the accuracy of translations from natural language to logical formulas. Current methods predominantly rely on prompt engineering, which is difficult to scale across different domains and input formats. Drawing inspiration from the success of fine-tuning in other model adaptation and alignment applications, we propose a fine-tuning-based Stratified Consistency Distillation approach: (1) We generate K logical translations per input using a frontier LLM and cluster them by semantic equivalence (2) Based on the entropy level, we apply majority voting (low entropy), LLM-as-a-Judge (medium entropy), or unification/abstention (high entropy), and (3) fine-tune a smaller model using the selected pseudo-labels. Our experiments show significant and consistent improvements in both Pass@K and our novel Equivalent Logical Similarity metrics, demonstrating the potential of advancing logical translation through consistency distillation.