mlsanityData to 7 Oct 2026

Aligning a medium-size GPT model in English to a small closed domain in Spanish

arXiv 2303.17649 ↗First posted 30 Mar 2023cs.CL cs.LGcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Reward modeling and RLHF

Abstract (arXiv)

In this paper, we propose a methodology to align a medium-sized GPT model, originally trained in English for an open domain, to a small closed domain in Spanish. The application for which the model is finely tuned is the question answering task. To achieve this we also needed to train and implement another neural network (which we called the reward model) that could score and determine whether an answer is appropriate for a given question. This component served to improve the decoding and generation of the answers of the system. Numerical metrics such as BLEU and perplexity were used to evaluate the model, and human judgment was also used to compare the decoding technique with others. Finally, the results favored the proposed method, and it was determined that it is feasible to use a reward model to align the generation of responses.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reward modeling and RLHF, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 6 of its 13 references that are in the map. First posted on arXiv 30 Mar 2023.

Builds on

6 of 13 references.

Closest ancestors · 1

Same direction, newest first

  1. 2019Fine-Tuning Language Models from Human Preferences18 Sep

Other references · 5

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
  5. 2021Smaaprat: DialoGPT for Natural Language Generation of Swedish Dialogue by Transfer Learningcited by 11

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Language Model Knowledge Distillation for Efficient Question Answering in Spanish
  2. 2023Reward Collapse in Aligning Large Language Models
  3. 2024On the Low-Rank Parametrization of Reward Models for Controlled Language Generation
  4. 2024Prompt Perturbation in Retrieval-Augmented Generation based Large Language Models
  5. 2024Rethinking the Role of Proxy Rewards in Language Model Alignment
  6. 2025CALM: Unleashing the Cross-Lingual Self-Aligning Ability of Language Model Question Answering
  7. 2025Sentence-level Reward Model can Generalize Better for Aligning LLM from Human Preference
  8. 2026Robust Reward Modeling for Large Language Models via Causal Decomposition