# Reinforcement learning to improve large language model-based automated code compliance systems

arXiv: https://arxiv.org/abs/2606.22402 · HTML: https://mlsanity.com/paper/2606.22402/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 45 references that are in the map.
- First posted on arXiv 21 Jun 2026.

## Builds on (most cited first, 16 of 16)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [Fine-tuning Language Models for Factuality](https://mlsanity.com/paper/2311.08401/) (2023)
- [Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputs](https://mlsanity.com/paper/2407.01082/) (2024)
- [Med-R1: Reinforcement Learning for Generalizable Medical Reasoning in Vision-Language Models](https://mlsanity.com/paper/2503.13939/) (2025)
- [Factually Consistent Summarization via Reinforcement Learning with Textual Entailment Feedback](https://mlsanity.com/paper/2306.00186/) (2023)
- [Uncertainty-Penalized Reinforcement Learning from Human Feedback with Diverse Reward LoRA Ensembles](https://mlsanity.com/paper/2401.00243/) (2023)
- [Inverse Scaling in Test-Time Compute](https://mlsanity.com/paper/2507.14417/) (2025)
- [Using Large Language Models for the Interpretation of Building Regulations](https://mlsanity.com/paper/2407.21060/) (2024)
- [Translating Regulatory Clauses into Executable Codes for Building Design Checking via Large Language Model Driven Function Matching and Composing](https://mlsanity.com/paper/2308.08728/) (2023)
- [A Text Classification-Based Approach for Evaluating and Enhancing the Machine Interpretability of Building Codes](https://mlsanity.com/paper/2309.14374/) (2023)
- [LLM-assisted Graph-RAG Information Extraction from IFC Data](https://mlsanity.com/paper/2504.16813/) (2025)
- [Optimizing Performance: How Compact Models Match or Exceed GPT's Classification Capabilities through Fine-Tuning](https://mlsanity.com/paper/2409.11408/) (2024)
- [LLM attribution analysis across different fine-tuning strategies and model scales for automated code compliance](https://mlsanity.com/paper/2604.15589/) (2026)

## Abstract (arXiv)

Large language model (LLM)-based approaches for automated code compliance (ACC) of building regulations are prone to generating incorrect and hallucinated computer-processable rules. This paper introduces P4IR, a two-stage framework that uses supervised fine-tuning (SFT) to instill domain knowledge in an LLM, followed by Group Relative Policy Optimization (GRPO) to improve the accuracy of the generated intermediate representations in the form of high-level code skeletons. The framework achieved reductions of up to 23.8% and 38.6% in tree edit distance and token-level Levenshtein distance respectively, relative to the SFT baselines. Comparative analysis demonstrates that this approach in a zero-shot setting outperforms leading LLMs in both code structure and semantics, specifically Claude Opus and Sonnet 4.5, GPT-5.2, Qwen-3-Max, and GLM-4.7, evaluated via few-shot prompting. Additionally, the GRPO stage produced a small yet statistically significant reduction in false positives. By combining SFT with GRPO to optimize directly for domain-specific objectives, this approach offers a path toward more accurate and reliable LLM-based ACC systems.