# Optimizing Retrieval Augmented Generation for Object Constraint Language

arXiv: https://arxiv.org/abs/2505.13129 · HTML: https://mlsanity.com/paper/2505.13129/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation.
- Builds on 14 of its 19 references that are in the map.
- First posted on arXiv 19 May 2025.

## Placed in

- Retrieval-augmented generation / [RAG pipeline design and evaluation](https://mlsanity.com/rag/rag-pipeline-design-and-evaluation/)

## Builds on (most cited first, 14 of 14)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence](https://mlsanity.com/paper/2401.14196/) (2024)
- [SPLADE: Sparse Lexical and Expansion Model for First Stage Ranking](https://mlsanity.com/paper/2107.05720/) (2021)
- [Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMs](https://mlsanity.com/paper/2312.05934/) (2023)
- [Large Language Models with Controllable Working Memory](https://mlsanity.com/paper/2211.05110/) (2022)
- [RAG vs Fine-tuning: Pipelines, Tradeoffs, and a Case Study on Agriculture](https://mlsanity.com/paper/2401.08406/) (2024)
- [Enhancing LLM Factual Accuracy with RAG to Counter Hallucinations: A Case Study on Domain-Specific Queries in Private Knowledge-Bases](https://mlsanity.com/paper/2403.10446/) (2024)
- [Fine Tuning vs. Retrieval Augmented Generation for Less Popular Knowledge](https://mlsanity.com/paper/2403.01432/) (2024)
- [On the Effectiveness of Large Language Models in Domain-Specific Code Generation](https://mlsanity.com/paper/2312.01639/) (2023)
- [L2CEval: Evaluating Language-to-Code Generation Capabilities of Large Language Models](https://mlsanity.com/paper/2309.17446/) (2023)
- [On Codex Prompt Engineering for OCL Generation: An Empirical Study](https://mlsanity.com/paper/2303.16244/) (2023)
- [PathOCL: Path-Based Prompt Augmentation for OCL Generation with GPT-4](https://mlsanity.com/paper/2405.12450/) (2024)

## Built on by (most cited first, 4 of 4)

- [Survey of GenAI for Automotive Software Development: From Requirements to Executable Code](https://mlsanity.com/paper/2507.15025/) (2025)
- [GenAI for Automotive Software Development: From Requirements to Wheels](https://mlsanity.com/paper/2507.18223/) (2025)
- [Leveraging LLMs to support co-evolution between definitions and instances of textual DSLs: A Systematic Evaluation](https://mlsanity.com/paper/2602.11904/) (2026)
- [Why Do LLMs Fail at OCL Generation? A Graph Reasoning Perspective](https://mlsanity.com/paper/2609.26122/) (2026)

## Abstract (arXiv)

The Object Constraint Language (OCL) is essential for defining precise constraints within Model-Based Systems Engineering (MBSE). However, manually writing OCL rules is complex and time-consuming. This study explores the optimization of Retrieval-Augmented Generation (RAG) for automating OCL rule generation, focusing on the impact of different retrieval strategies. We evaluate three retrieval approaches unicodex2013 BM25 (lexical-based), BERT-based (semantic retrieval), and SPLADE (sparse-vector retrieval) unicodex2013 analyzing their effectiveness in providing relevant context for a large language model. To further assess our approach, we compare and benchmark our retrieval-optimized generation results against PathOCL, a state-of-the-art graph-based method. We directly compare BM25, BERT, and SPLADE retrieval methods with PathOCL to understand how different retrieval methods perform for a unified evaluation framework. Our experimental results, focusing on retrieval-augmented generation, indicate that while retrieval can enhance generation accuracy, its effectiveness depends on the retrieval method and the number of retrieved chunks (k). BM25 underperforms the baseline, whereas semantic approaches (BERT and SPLADE) achieve better results, with SPLADE performing best at lower k values. However, excessive retrieval with high k parameter can lead to retrieving irrelevant chunks which degrades model performance. Our findings highlight the importance of optimizing retrieval configurations to balance context relevance and output consistency. This research provides insights into improving OCL rule generation using RAG and underscores the need for tailoring retrieval.