mlsanityData to 7 Oct 2026

CHAI for LLMs: Improving Code-Mixed Translation in Large Language Models through Reinforcement Learning with AI Feedback

arXiv 2411.09073 ↗First posted 13 Nov 2024cs.CL cs.AI cs.LGcited by 5

On the map: LLM alignment / RLHF / preference optimisation → Process reward models for reasoning

Abstract (arXiv)

Large language models (LLMs) show strong performance across many tasks but remain weak at understanding code-mixed (CM) language. Despite this limitation, improving LLMs for CM tasks has received little attention. To address this gap, we propose CHAI, a general-purpose framework for enhancing LLM performance on CM tasks, focusing on CM translation. CHAI leverages four key ideas. First, we investigate the use of LLMs as annotators to address the scarcity of high-quality CM datasets. Second, we leverage these LLM annotations to generate large-scale preference data and apply reinforcement learning from AI feedback (RLAIF) to improve CM translation. Third, we incorporate LLM-generated domain knowledge as a constitution, enabling iterative response refinement. Fourth, we perform extensive evaluations on real-world datasets and settings. Results show that CHAI-powered LLMs outperform state-of-the-art open-source models by 68.45% on average in human-adjudicated win rate on CM translation tasks. This work is a step toward more inclusive open-source code-mixed LLMs.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Process reward models for reasoning, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 30 of its 67 references that are in the map. First posted on arXiv 13 Nov 2024.

Builds on

30 of 67 references.

References · 30

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2024The Llama 3 Herd of Modelscited by 14,192
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  6. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  7. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  8. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  9. 2019Cross-lingual Language Model Pretrainingcited by 1,088
  10. 2020COMET: A Neural Framework for MT Evaluationcited by 1,024
  11. 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 615
  12. 2019Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialogcited by 283
18 more
  1. 2023Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questionscited by 249
  2. 2024Secrets of RLHF in Large Language Models Part II: Reward Modelingcited by 170
  3. 2020Low-resource Languages: A Review of Past Work and Future Challengescited by 90
  4. 2023Is ChatGPT the Ultimate Programming Assistant -- How far is it?cited by 72
  5. 2023Multilingual Large Language Models Are Not (Yet) Code-Switcherscited by 65
  6. 2020SemEval-2020 Task 9: Overview of Sentiment Analysis of Code-Mixed Tweetscited by 62
  7. 2024Self-Boosting Large Language Models with Synthetic Preference Datacited by 40
  8. 2020PHINC: A Parallel Hinglish Social Media Code-Mixed Corpus for Machine Translationcited by 38
  9. 2024Improving Machine Translation with Human Feedback: An Exploration of Quality Estimation as a Reward Modelcited by 33
  10. 2021From Machine Translation to Code-Switching: Generating High-Quality Code-Switched Textcited by 29
  11. 2021HinGE: A Dataset for Generation and Evaluation of Code-Mixed Hinglish Textcited by 28
  12. 2023Can ChatGPT be used to generate scientific hypotheses?cited by 21
  13. 2021Challenges and Limitations with the Metrics Measuring the Complexity of Code-Mixed Textcited by 18
  14. 2024MetaMetrics: Calibrating Metrics For Generation Tasks Using Human Preferencescited by 15
  15. 2024Advancing Translation Preference Modeling with RLHF: A Step Towards Cost-Effective Solutioncited by 13
  16. 2024ALaRM: Align Language Models via Hierarchical Rewards Modelingcited by 11
  17. 2024Code-Mixer Ya Nahi: Novel Approaches to Measuring Multilingual LLMs' Code-Mixing Capabilitiescited by 7
  18. 2024Multilingual Controlled Generation And Gold-Standard-Agnostic Evaluation of Code-Mixed Sentencescited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 1.

5 papers.

  1. 2025Beyond Monolingual Assumptions: A Survey of Code-Switched NLP in the Era of Large Language Models across Modalitiescited by 4
  2. 2025Gradual Code-Switching as Inference-Time Cross-Lingual Representational Alignment for LLMscited by 3
  3. 2025The Hardness of Achieving Impact in AI for Social Impact Research: A Ground-Level View of Challenges & Opportunitiescited by 1
  4. 2025Towards Fine-Grained Code-Switch Speech Translation with Semantic Space Alignmentcited by 1
  5. 2026Code Mixologist : A Practitioner's Guide to Building Code-Mixed LLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learning
  2. 2025MT-R1-Zero: Advancing LLM-based Machine Translation via R1-Zero-like Reinforcement Learning
  3. 2026Translate-R1: Cost-Aware Translation Tool Use via Reinforcement Learning
  4. 2025From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligence
  5. 2026CodePivot: Bootstrapping Multilingual Transpilation in LLMs via Reinforcement Learning without Parallel Corpora
  6. 2026Mending the Holes: Mitigating Reward Hacking in Reinforcement Learning for Multilingual Translation
  7. 2024A critical review of methods and challenges in large language models
  8. 2024Large Language Model Evaluation Via Multi AI Agents: Preliminary results