# Embedding Retrofitting: Data Engineering for better RAG

arXiv: https://arxiv.org/abs/2601.15298 · HTML: https://mlsanity.com/paper/2601.15298/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 17 references that are in the map.
- First posted on arXiv 6 Jan 2026.

## Builds on (most cited first, 9 of 9)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [Improving language models by retrieving from trillions of tokens](https://mlsanity.com/paper/2112.04426/) (2021)
- [Counter-fitting Word Vectors to Linguistic Constraints](https://mlsanity.com/paper/1603.00892/) (2016)
- [Morph-fitting: Fine-Tuning Word Vector Spaces with Simple Language-Specific Rules](https://mlsanity.com/paper/1706.00377/) (2017)
- [Mitigating Hallucination with ZeroG: An Advanced Knowledge Management Engine](https://mlsanity.com/paper/2411.05936/) (2024)

## Built on by (most cited first, 1 of 1)

- [Trustworthy Multi-Agent Systems: Mitigating Semantic Drift with the Argent Signaling Protocol](https://mlsanity.com/paper/2606.19356/) (2026)

## Abstract (arXiv)

Embedding retrofitting adjusts pre-trained word vectors using knowledge graph constraints to improve domain-specific retrieval. However, the effectiveness of retrofitting depends critically on knowledge graph quality, which in turn depends on text preprocessing. This paper presents a data engineering framework that addresses data quality degradation from annotation artifacts in real-world corpora. The analysis shows that hashtag annotations inflate knowledge graph density, leading to creating spurious edges that corrupt the retrofitting objective. On noisy graphs, all retrofitting techniques produce statistically significant degradation (-3.5% to -5.2%, p<0.05). After preprocessing, acrshortewma retrofitting achieves +6.2% improvement (p=0.0348) with benefits concentrated in quantitative synthesis questions (+33.8% average). The gap between clean and noisy preprocessing (10%+ swing) exceeds the gap between algorithms (3%), establishing preprocessing quality as the primary determinant of retrofitting success.