# Billiger.de Products: A Bilingual Entity Matching Benchmark

arXiv: https://arxiv.org/abs/2609.37713 · HTML: https://mlsanity.com/paper/2609.37713/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 40 references that are in the map.
- First posted on arXiv 29 Sep 2026.

## Builds on (most cited first, 18 of 18)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Can Foundation Models Wrangle Your Data?](https://mlsanity.com/paper/2205.09911/) (2022)
- [Deep Entity Matching with Pre-Trained Language Models](https://mlsanity.com/paper/2004.00584/) (2020)
- [Neural Networks for Entity Matching: A Survey](https://mlsanity.com/paper/2010.11075/) (2020)
- [Match, Compare, or Select? An Investigation of Large Language Models for Entity Matching](https://mlsanity.com/paper/2405.16884/) (2024)
- [Entity Matching using Large Language Models](https://mlsanity.com/paper/2310.11244/) (2023)
- [Supervised Contrastive Learning for Product Matching](https://mlsanity.com/paper/2202.02098/) (2022)
- [(Almost) All of Entity Resolution](https://mlsanity.com/paper/2008.04443/) (2020)
- [Using ChatGPT for Entity Matching](https://mlsanity.com/paper/2305.03423/) (2023)
- [WDC Products: A Multi-Dimensional Entity Matching Benchmark](https://mlsanity.com/paper/2301.09521/) (2023)
- [Machamp: A Generalized Entity Matching Benchmark](https://mlsanity.com/paper/2106.08455/) (2021)
- [Fine-tuning Large Language Models for Entity Matching](https://mlsanity.com/paper/2409.08185/) (2024)
- [Bridging the Gap between Reality and Ideality of Entity Matching: A Revisiting and Benchmark Re-Construction](https://mlsanity.com/paper/2205.05889/) (2022)
- [Alaska: A Flexible Benchmark for Data Integration Tasks](https://mlsanity.com/paper/2101.11259/) (2021)
- [Cross-Language Learning for Entity Matching](https://mlsanity.com/paper/2110.03338/) (2021)
- [ProMap: Datasets for Product Mapping in E-commerce](https://mlsanity.com/paper/2309.06882/) (2023)
- [Multilingual Transformers for Product Matching -- Experiments and a New Benchmark in Polish](https://mlsanity.com/paper/2205.15712/) (2022)

## Abstract (arXiv)

Existing product matching benchmarks primarily contain English-language product data and are often dominated by a single product category, such as electronics. This paper introduces Billiger.de Products, a bilingual German and English entity matching benchmark covering thirteen consumer product categories, including difficult-to-handle categories such as clothing and furniture. The benchmark data originates from the German price comparison platform billiger.de. Following the design of WDC Products, the benchmark offers multiple variants that differ in the fraction of corner cases, the size of the development set, and the fraction of entities unseen during training. An aligned English translation of every offer keeps all pairs, splits, and labels fixed, while cross-language test sets combine German and English records within individual pairs. We validate the benchmark using six supervised matchers and zero-shot GPT-5.2 on both language versions and the cross-language test sets. The validation shows the difficulty of the benchmark. The comparison of the results on the English version of the benchmark to the results on the German version shows that most matchers score on average higher on the English version. The difference is largest for RoBERTa and HierGAT, while the zero-shot LLM runs are largely insensitive to the language. Comparing the F1 scores achieved by PLM-based matchers on the English version of Billiger.de Products with their performance on existing English-language benchmarks, such as WDC Products and Abt-Buy, shows that Billiger.de Products is more difficult than these benchmarks.