# Multi-Lingual Malaysian Embedding: Leveraging Large Language Models for Semantic Representations

arXiv: https://arxiv.org/abs/2402.03053 · HTML: https://mlsanity.com/paper/2402.03053/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Generation and context integration, one of 15 directions in Retrieval-augmented generation.
- Builds on 2 of its 3 references that are in the map.
- First posted on arXiv 5 Feb 2024.

## Placed in

- Retrieval-augmented generation / [Generation and context integration](https://mlsanity.com/rag/generation-and-context-integration/)

## Builds on (most cited first, 2 of 2)

- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [Large Malaysian Language Model Based on Mistral for Enhanced Local Language Understanding](https://mlsanity.com/paper/2401.13565/) (2024)

## Built on by (most cited first, 2 of 2)

- [LLMs are Also Effective Embedding Models: An In-depth Overview](https://mlsanity.com/paper/2412.12591/) (2024)
- [Retrieval with Multiple Query Vectors through Anomalous Pattern Detection](https://mlsanity.com/paper/2605.01965/) (2026)

## Abstract (arXiv)

In this work, we present a comprehensive exploration of finetuning Malaysian language models, specifically Llama2 and Mistral, on embedding tasks involving negative and positive pairs. We release two distinct models tailored for Semantic Similarity and Retrieval-Augmented Generation (RAG). For Semantic Similarity, our 600 million parameter Llama2 model outperforms OpenAI text-embedding-ada-002 across all recall@k metrics for b.cari.com.my, c.cari.com.my, Malay news, and Malaysian Twitter test sets. In the realm of RAG models, our approach proves competitive with OpenAI text-embedding-ada-002 in the Malaysian context. Notably, our 2 billion parameter Llama2 model achieves superior Recall@5, Recall@10 for the "Melayu" keyword research papers dataset and excels in Recall@3, Recall@5, and Recall@10 for the lom.agc.gov.my dataset. These findings underscore the effectiveness of our finetuning strategy and highlight the performance gains in both Semantic Similarity and RAG tasks. All models released at https://huggingface.co/collections/mesolitica/malaysian-embedding-6523612bfe5881ad35f81b99