mlsanityData to 7 Oct 2026

One Instruction Does Not Fit All: How Well Do Embeddings Align Personas and Instructions in Low-Resource Indian Languages?

arXiv 2601.10205 ↗First posted 15 Jan 2026cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Aligning multilingual assistants with culturally grounded user preferences is essential for serving India's linguistically diverse population of over one billion speakers across multiple scripts. However, existing benchmarks either focus on a single language or conflate retrieval with generation, leaving open the question of whether current embedding models can encode persona-instruction compatibility without relying on response synthesis. We present a unified benchmark spanning 12 Indian languages and four evaluation tasks: monolingual and cross-lingual persona-to-instruction retrieval, reverse retrieval from instruction to persona, and binary compatibility classification. Eight multilingual embedding models are evaluated in a frozen-encoder setting with a thin logistic regression head for classification. E5-Large-Instruct achieves the highest Recall@1 of 27.4% on monolingual retrieval and 20.7% on cross-lingual transfer, while BGE-M3 leads reverse retrieval at 32.1% Recall@1. For classification, LaBSE attains 75.3% AUROC with strong calibration. These findings offer practical guidance for model selection in Indic multilingual retrieval and establish reproducible baselines for future workfootnoteCode, datasets, and models are publicly available at https://github.com/aryashah2k/PI-Indic-Align.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 32 of its 49 references that are in the map. First posted on arXiv 15 Jan 2026.

Builds on

32 of 49 references.

References · 32

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  2. 2017On Calibration of Modern Neural Networkscited by 4,860
  3. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  4. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  5. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  6. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  7. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  8. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,050
  9. 2018Personalizing Dialogue Agents: I have a dog, do you have pets too?cited by 820
  10. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 738
  11. 2020TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languagescited by 486
  12. 2021Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Modelscited by 462
20 more
  1. 2024Scaling Synthetic Data Creation with 1,000,000,000 Personascited by 397
  2. 2024Multilingual E5 Text Embeddings: A Technical Reportcited by 357
  3. 2022One Embedder, Any Task: Instruction-Finetuned Text Embeddingscited by 329
  4. 2020Calibration of Pre-trained Transformerscited by 307
  5. 2024mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrievalcited by 233
  6. 2024Generative Representational Instruction Tuningcited by 227
  7. 2021MuRIL: Multilingual Representations for Indian Languagescited by 177
  8. 2021XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluationcited by 167
  9. 2023IndicTrans2: Towards High-Quality and Accessible Machine Translation Models for all 22 Scheduled Indian Languagescited by 149
  10. 2018Training Millions of Personalized Dialogue Agentscited by 126
  11. 2021Mr. TyDi: A Multi-lingual Benchmark for Dense Retrievalcited by 118
  12. 2021mMARCO: A Multilingual Version of the MS MARCO Passage Ranking Datasetcited by 116
  13. 2022Towards Leaving No Indic Language Behind: Building Monolingual Corpora, Benchmark and Models for Indic Languagescited by 99
  14. 2024FollowIR: Evaluating and Teaching Information Retrieval Models to Follow Instructionscited by 67
  15. 2022Making a MIRACL: Multilingual Information Retrieval Across a Continuum of Languagescited by 47
  16. 2020LAReQA: Language-agnostic answer retrieval from a multilingual poolcited by 41
  17. 2022L3Cube-HindBERT and DevBERT: Pre-Trained BERT Transformer models for Devanagari based Hindi and Marathi Languagescited by 35
  18. 2024Answer is All You Need: Instruction-following Text Embedding via Answering the Questioncited by 26
  19. 2019Dually Interactive Matching Network for Personalized Response Selection in Retrieval-Based Chatbotscited by 23
  20. 2024Learning Retrieval Augmentation for Personalized Dialogue Generationcited by 18

Built on by

Not cited yet.