# CHILL at SemEval-2025 Task 2: You Can't Just Throw Entities and Hope -- Make Your LLM to Get Them Right

arXiv: https://arxiv.org/abs/2506.13070 · HTML: https://mlsanity.com/paper/2506.13070/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 20 references that are in the map.
- First posted on arXiv 16 Jun 2025.

## Builds on (most cited first, 10 of 10)

- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [COMET: A Neural Framework for MT Evaluation](https://mlsanity.com/paper/2009.09025/) (2020)
- [Autoregressive Entity Retrieval](https://mlsanity.com/paper/2010.00904/) (2020)
- [GEMBA-MQM: Detecting Translation Quality Error Spans with GPT-4](https://mlsanity.com/paper/2310.13988/) (2023)
- [Efficient One-Pass End-to-End Entity Linking for Questions](https://mlsanity.com/paper/2010.02413/) (2020)
- [Leveraging GPT-4 for Automatic Translation Post-Editing](https://mlsanity.com/paper/2305.14878/) (2023)
- [DEEP: DEnoising Entity Pre-training for Neural Machine Translation](https://mlsanity.com/paper/2111.07393/) (2021)
- [Generative Relation Linking for Question Answering over Knowledge Bases](https://mlsanity.com/paper/2108.07337/) (2021)
- [SPARKLE: Enhancing SPARQL Generation with Direct KG Integration in Decoding](https://mlsanity.com/paper/2407.01626/) (2024)

## Abstract (arXiv)

In this paper, we describe our approach for the SemEval 2025 Task 2 on Entity-Aware Machine Translation (EA-MT). Our system aims to improve the accuracy of translating named entities by combining two key approaches: Retrieval Augmented Generation (RAG) and iterative self-refinement techniques using Large Language Models (LLMs). A distinctive feature of our system is its self-evaluation mechanism, where the LLM assesses its own translations based on two key criteria: the accuracy of entity translations and overall translation quality. We demonstrate how these methods work together and effectively improve entity handling while maintaining high-quality translations.