# Standardizing Knowledge Engineering Practices with a Reference Architecture

arXiv: https://arxiv.org/abs/2404.03624 · HTML: https://mlsanity.com/paper/2404.03624/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 96 references that are in the map.
- First posted on arXiv 4 Apr 2024.

## Builds on (most cited first, 17 of 17)

- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [Trustworthy AI: From Principles to Practices](https://mlsanity.com/paper/2110.01167/) (2021)
- [Leveraging Pre-trained Large Language Models to Construct and Utilize World Models for Model-based Task Planning](https://mlsanity.com/paper/2305.14909/) (2023)
- [OpenAlex: A fully-open index of scholarly works, authors, venues, institutions, and concepts](https://mlsanity.com/paper/2205.01833/) (2022)
- [A Review on Language Models as Knowledge Bases](https://mlsanity.com/paper/2204.06031/) (2022)
- [PyTorch-BigGraph: A Large-scale Graph Embedding System](https://mlsanity.com/paper/1903.12287/) (2019)
- [CSKG: The CommonSense Knowledge Graph](https://mlsanity.com/paper/2012.11490/) (2020)
- [A Decade of Knowledge Graphs in Natural Language Processing: A Survey](https://mlsanity.com/paper/2210.00105/) (2022)
- [Modular Design Patterns for Hybrid Learning and Reasoning Systems: a taxonomy, patterns and use cases](https://mlsanity.com/paper/2102.11965/) (2021)
- [A Study of the Quality of Wikidata](https://mlsanity.com/paper/2107.00156/) (2021)
- [A Boxology of Design Patterns for Hybrid Learning and Reasoning Systems](https://mlsanity.com/paper/1905.12389/) (2019)
- [Getting from Generative AI to Trustworthy AI: What LLMs might learn from Cyc](https://mlsanity.com/paper/2308.04445/) (2023)
- [KGTK: A Toolkit for Large Knowledge Graph Manipulation and Analysis](https://mlsanity.com/paper/2006.00088/) (2020)
- [A Semantic Web Technology Index](https://mlsanity.com/paper/2201.07034/) (2022)
- [The Pipeline for the Continuous Development of Artificial Intelligence Models -- Current State of Research and Practice](https://mlsanity.com/paper/2301.09001/) (2023)
- [Describing and Organizing Semantic Web and Machine Learning Systems in the SWeMLS-KG](https://mlsanity.com/paper/2303.15113/) (2023)
- [Identifying and Consolidating Knowledge Engineering Requirements](https://mlsanity.com/paper/2306.15124/) (2023)

## Abstract (arXiv)

Knowledge engineering is the process of creating and maintaining knowledge-producing systems. Throughout the history of computer science and AI, knowledge engineering workflows have been widely used given the importance of high-quality knowledge for reliable intelligent agents. Meanwhile, the scope of knowledge engineering, as apparent from its target tasks and use cases, has been shifting, together with its paradigms such as expert systems, semantic web, and language modeling. The intended use cases and supported user requirements between these paradigms have not been analyzed globally, as new paradigms often satisfy prior pain points while possibly introducing new ones. The recent abstraction of systemic patterns into a boxology provides an opening for aligning the requirements and use cases of knowledge engineering with the systems, components, and software that can satisfy them best. This paper proposes a vision of harmonizing the best practices in the field of knowledge engineering by leveraging the software engineering methodology of creating reference architectures. We describe how a reference architecture can be iteratively designed and implemented to associate user needs with recurring systemic patterns, building on top of existing knowledge engineering workflows and boxologies. We provide a six-step roadmap that can enable the development of such an architecture, providing an initial design and outcome of the definition of architectural scope, selection of information sources, and analysis. We expect that following through on this vision will lead to well-grounded reference architectures for knowledge engineering, will advance the ongoing initiatives of organizing the neurosymbolic knowledge engineering space, and will build new links to the software architectures and data science communities.