# Matching with Transformers in MELT

arXiv: https://arxiv.org/abs/2109.07401 · HTML: https://mlsanity.com/paper/2109.07401/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 20 references that are in the map.
- First posted on arXiv 15 Sep 2021.

## Builds on (most cited first, 10 of 10)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Tune: A Research Platform for Distributed Model Selection and Training](https://mlsanity.com/paper/1807.05118/) (2018)
- [Population Based Training of Neural Networks](https://mlsanity.com/paper/1711.09846/) (2017)
- [The Knowledge Graph Track at OAEI -- Gold Standards, Baselines, and the Golden Hammer Bias](https://mlsanity.com/paper/2002.10283/) (2020)
- [Supervised Ontology and Instance Matching with MELT](https://mlsanity.com/paper/2009.11102/) (2020)
- [Visual Analysis of Ontology Matching Results with the MELT Dashboard](https://mlsanity.com/paper/2004.12628/) (2020)

## Built on by (most cited first, 2 of 2)

- [Contextualized Structural Self-supervised Learning for Ontology Matching](https://mlsanity.com/paper/2310.03840/) (2023)
- [KERMIT -- A Transformer-Based Approach for Knowledge Graph Matching](https://mlsanity.com/paper/2204.13931/) (2022)

## Abstract (arXiv)

One of the strongest signals for automated matching of ontologies and knowledge graphs are the textual descriptions of the concepts. The methods that are typically applied (such as character- or token-based comparisons) are relatively simple, and therefore do not capture the actual meaning of the texts. With the rise of transformer-based language models, text comparison based on meaning (rather than lexical features) is possible. In this paper, we model the ontology matching task as classification problem and present approaches based on transformer models. We further provide an easy to use implementation in the MELT framework which is suited for ontology and knowledge graph matching. We show that a transformer-based filter helps to choose the correct correspondences given a high-recall alignment and already achieves a good result with simple alignment post-processing methods.