# LLM attribution analysis across different fine-tuning strategies and model scales for automated code compliance

arXiv: https://arxiv.org/abs/2604.15589 · HTML: https://mlsanity.com/paper/2604.15589/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 8 references that are in the map.
- First posted on arXiv 16 Apr 2026.

## Builds on (most cited first, 2 of 2)

- [Using Captum to Explain Generative Language Models](https://mlsanity.com/paper/2312.05491/) (2023)
- [Translating Regulatory Clauses into Executable Codes for Building Design Checking via Large Language Model Driven Function Matching and Composing](https://mlsanity.com/paper/2308.08728/) (2023)

## Built on by (most cited first, 1 of 1)

- [Reinforcement learning to improve large language model-based automated code compliance systems](https://mlsanity.com/paper/2606.22402/) (2026)

## Abstract (arXiv)

Existing research on large language models (LLMs) for automated code compliance has primarily focused on performance, treating the models as black boxes and overlooking how training decisions affect their interpretive behavior. This paper addresses this gap by employing a perturbation-based attribution analysis to compare the interpretive behaviors of LLMs across different fine-tuning strategies such as full fine-tuning (FFT), low-rank adaptation (LoRA) and quantized LoRA fine-tuning, as well as the impact of model scales which include varying LLM parameter sizes. Our results show that FFT produces attribution patterns that are statistically different and more focused than those from parameter-efficient fine-tuning methods. Furthermore, we found that as model scale increases, LLMs develop specific interpretive strategies such as prioritizing numerical constraints and rule identifiers in the building text, albeit with performance gains in semantic similarity of the generated and reference computer-processable rules plateauing for models larger than 7B. This paper provides crucial insights into the explainability of these models, taking a step toward building more transparent LLMs for critical, regulation-based tasks in the Architecture, Engineering, and Construction industry.