Early-Stage Product Line Validation Using LLMs: A Study on Semi-Formal Blueprint Analysis
Not on the map. Why?
Abstract (arXiv)
We study whether Large Language Models (LLMs) can perform feature model analysis operations (AOs) directly on semi-formal textual blueprints, i.e., concise constrained-language descriptions of feature hierarchies and constraints, enabling early validation in Software Product Line scoping. Using 12 state-of-the-art LLMs and 16 standard AOs, we compare their outputs against the solver-based oracle FLAMA. Results show that reasoning-optimized models (e.g., Grok 4 Fast Reasoning, Gemini 2.5 Pro) achieve 88-89% average accuracy across all evaluated blueprints and operations, approaching solver correctness. We identify systematic errors in structural parsing and constraint reasoning, and highlight accuracy-cost trade-offs that inform model selection. These findings position LLMs as lightweight assistants for early variability validation.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 8 of its 34 references that are in the map. First posted on arXiv 22 Apr 2026.
Builds on
8 of 34 references.
References · 8
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
- 2022Towards Reasoning in Large Language Models: A Surveycited by 593
- 2025ZebraLogic: On the Scaling Limits of LLMs for Logical Reasoningcited by 186
- 2024LogicBench: Towards Systematic Evaluation of Logical Reasoning Ability of Large Language Modelscited by 67
- 2023Leveraging Large Language Models to Generate Answer Set Programscited by 33
- 2024Can Transformers Reason Logically? A Study in SAT Solvingcited by 16
- 2024Do Large Language Models Understand Logic or Just Mimick Context?cited by 10
Built on by
Not cited yet.