IDRAAK: From Multi-Agent NLP to Few-Shot Prompting for Semantic Drift Detection in Technical Requirements
On the map: LLM agents and tool use → Software engineering coding agents
Abstract (arXiv)
Translating technical requirements across languages can introduce semantic drift, altering numerical constraints, polarities, modalities, or other specification-critical meaning. IDRAAK is presented as an interpretable framework for detecting such drift using a language-independent Semantic Requirement Representation (SRR), with six detection workflows evaluated, ranging from deterministic comparison to multi-agent verification and few-shot prompting. On 890 synthetic perturbations across 300 requirements from 10 engineering domains, a single LLM call with six few-shot examples achieves MCC=0.888 and F1=0.983, outperforming the evaluated structured and multi-stage alternatives. Further evaluation on PAWS-X (805 pairs, 5 languages) and XNLI (700 pairs, 7 languages) exposes complementary strengths and limitations of structured and LLM-based approaches. Deterministic SRR comparison performs strongly on technical requirements (F1=0.898) but poorly on general-domain text (F1=0.012), while structured evidence improves performance on adversarial paraphrases. Post-hoc Platt scaling further improves confidence calibration. The results demonstrate that increased agentic complexity does not necessarily improve semantic-drift detection and that simple few-shot prompting can provide a strong and efficient alternative.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 25 of its 42 references that are in the map. First posted on arXiv 9 Aug 2026.
Builds on
25 of 42 references.
Closest ancestors · 1
Same direction, newest first
Other references · 24
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2023GPT-4 Technical Reportcited by 18,396
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
- 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
- 2017On Calibration of Modern Neural Networkscited by 4,960
- 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
- 2022Survey of Hallucination in Natural Language Generationcited by 2,075
- 2022Language Models (Mostly) Know What They Knowcited by 1,856
- 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,808
- 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
12 more
- 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,434
- 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,203
- 2021What Makes Good In-Context Examples for GPT-3?cited by 1,195
- 2020COMET: A Neural Framework for MT Evaluationcited by 1,033
- 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 1,001
- 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 915
- 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 900
- 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debatecited by 742
- 2019How multilingual is Multilingual BERT?cited by 645
- 2019PAWS: Paraphrase Adversaries from Word Scramblingcited by 402
- 2019PAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identificationcited by 264
- 2020Are All Languages Created Equal in Multilingual BERT?cited by 207
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026BT-APE: A Computationally Light Backtracking Approach to Automatic Prompt Engineering for Requirements Classification ↗
- 2026A Retrieval-Augmented Framework for Detecting and Resolving Pragmatic Ambiguities in Natural Language Requirements
- 2026Automating the Detection of Requirement Dependencies Using Large Language Models
- 2025Are Prompts All You Need? Evaluating Prompt-Based Large Language Models (LLM)s for Software Requirements Classification
- 2026Automatic Translation of Unstructured Requirements into Linear Temporal Logic through Large Language Models ↗
- 2026Evaluating LLM-Based Goal Extraction in Requirements Engineering: Prompting Strategies and Their Limitations
- 2025Prompt Engineering for Requirements Engineering: A Literature Review and Roadmap
- 2026Agent4RE: A Self-Refining Multi-agent Framework for End-to-End Software Requirements Engineering and Benchmarking