# Evaluating LLM-Based Goal Extraction in Requirements Engineering: Prompting Strategies and Their Limitations

arXiv: https://arxiv.org/abs/2604.22207 · HTML: https://mlsanity.com/paper/2604.22207/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 33 references that are in the map.
- First posted on arXiv 24 Apr 2026.

## Builds on (most cited first, 9 of 9)

- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Large Language Models are Few-shot Testers: Exploring LLM-based General Bug Reproduction](https://mlsanity.com/paper/2209.11515/) (2022)
- [What Did I Do Wrong? Quantifying LLMs' Sensitivity and Consistency to Prompt Engineering](https://mlsanity.com/paper/2406.12334/) (2024)
- [Autonomous Large Language Model Agents Enabling Intent-Driven Mobile GUI Testing](https://mlsanity.com/paper/2311.08649/) (2023)
- [The Impact of Prompt Programming on Function-Level Code Generation](https://mlsanity.com/paper/2412.20545/) (2024)
- [Large Language Models (LLMs) for Requirements Engineering (RE): A Systematic Literature Review](https://mlsanity.com/paper/2509.11446/) (2025)
- [Ethical Aspects of ChatGPT in Software Engineering Research](https://mlsanity.com/paper/2306.07557/) (2023)
- [AI based Multiagent Approach for Requirements Elicitation and Analysis](https://mlsanity.com/paper/2409.00038/) (2024)
- [Semantic API Alignment: Linking High-level User Goals to APIs](https://mlsanity.com/paper/2405.04236/) (2024)

## Abstract (arXiv)

Due to the textual and repetitive nature of many Requirements Engineering (RE) artefacts, Large Language Models (LLMs) have proven useful to automate their generation and processing. In this paper, we discuss a possible approach for automating the Goal-Oriented Requirements Engineering (GORE) process by extracting functional goals from software documentation through three phases: actor identification, high and low-level goal extraction. To implement these functionalities, we propose a chain of LLMs fed with engineered prompts. We experimented with different variants of in-context learning and measured the similarities between input data and in-context examples to better investigate their impact. Another key element is the generation-critic mechanism, implemented as a feedback loop involving two LLMs. Although the pipeline achieved 61% accuracy in low-level goal identification, the final stage, these results indicate the approach is best suited as a tool to accelerate manual extraction rather than as a full replacement. The feedback-loop mechanism with Zero-shot outperformed stand-alone Few-shot, with an ablation study suggesting that performance slightly degrades without the feedback cycle. However, we reported that the combination of the feedback mechanism with Few-shot does not deliver any advantage, possibly suggesting that the primary performance ceiling is the prompting strategy applied to the 'critic' LLM. Together with the refinement of both the quantity and quality of the Shot examples, future research will integrate Retrieval-Augmented Generation (RAG) and Chain-of-Thought (CoT) prompting to improve accuracy.