mlsanityData to 7 Oct 2026

BabyStories: Can Reinforcement Learning Teach Baby Language Models to Write Better Stories?

arXiv 2310.16681 ↗First posted 25 Oct 2023cs.CLcited by 7

On the map: LLM alignment / RLHF / preference optimisation → Reward modeling and RLHF

Abstract (arXiv)

Language models have seen significant growth in the size of their corpus, leading to notable performance improvements. Yet, there has been limited progress in developing models that handle smaller, more human-like datasets. As part of the BabyLM shared task, this study explores the impact of reinforcement learning from human feedback (RLHF) on language models pretrained from scratch with a limited training corpus. Comparing two GPT-2 variants, the larger model performs better in storytelling tasks after RLHF fine-tuning. These findings suggest that RLHF techniques may be more advantageous for larger models due to their higher learning and adaptation capacity, though more experiments are needed to confirm this finding. These insights highlight the potential benefits of RLHF fine-tuning for language models within limited data, enhancing their ability to maintain narrative focus and coherence while adhering better to initial instructions in storytelling tasks. The code for this work is publicly at https://github.com/Zephyr1022/BabyStories-UTSA.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Reward modeling and RLHF, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 22 of its 43 references that are in the map. First posted on arXiv 25 Oct 2023.

Builds on

22 of 43 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Model29 May
  2. 2023Perspectives on the Social Impacts of Reinforcement Learning with Human Feedback6 Mar
  3. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback12 Apr

Other references · 19

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  4. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  5. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  6. 2020Learning to summarize from human feedbackcited by 2,562
  7. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  8. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
  9. 2018Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidatescited by 632
  10. 2019BLiMP: The Benchmark of Linguistic Minimal Pairs for Englishcited by 553
  11. 2023TinyStories: How Small Can Language Models Be and Still Speak Coherent English?cited by 425
  12. 2016A Corpus and Evaluation Framework for Deeper Understanding of Commonsense Storiescited by 413
7 more
  1. 2015The Goldilocks Principle: Reading Children's Books with Explicit Memory Representationscited by 108
  2. 2018Transforming Question Answering Datasets Into Natural Language Inference Datasetscited by 99
  3. 2016Edinburgh Neural Machine Translation Systems for WMT 16cited by 76
  4. 2018A standardized Project Gutenberg corpus for statistical analysis of natural language and quantitative linguisticscited by 64
  5. 2020Quantifying Intimacy in Languagecited by 25
  6. 2017Capturing Reliable Fine-Grained Sentiment Associations by Crowdsourcing and Best-Worst Scalingcited by 23
  7. 2023Honey, I Shrunk the Language: Language Model Behavior at Reduced Scalecited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 4; 2026: 1.

7 papers.

  1. 2025Findings of the BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corporacited by 86
  2. 2024Your Transformer is Secretly Linearcited by 23
  3. 2025Once Upon a Time: Interactive Learning for Storytelling with Small Language Modelscited by 11
  4. 2025Advantage-Guided Distillation for Preference Alignment in Small Language Modelscited by 4
  5. 2024Mutual Enhancement of Large Language and Reinforcement Learning Models through Bi-Directional Feedback Mechanisms: A Planning Case Studycited by 3
  6. 2025CLASS-IT: Conversational and Lecture-Aligned Small-Scale Instruction Tuning for BabyLMscited by 0
  7. 2026Which Forms of Caregiver Feedback Support Grammar Learning? A Reinforcement-Learning Study of Child-Like Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Does RLHF Scale? Exploring the Impacts From Data, Model, and Method
  2. 2023Understanding the Effects of RLHF on LLM Generalisation and Diversity
  3. 2024Secrets of RLHF in Large Language Models Part II: Reward Modeling
  4. 2024Sequence to Sequence Reward Modeling: Improving RLHF by Language Feedback
  5. 2024Findings of the Second BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corpora
  6. 2024The Accuracy Paradox in RLHF: When Better Reward Models Don't Yield Better Language Models
  7. 2024Understanding and Alleviating Memory Consumption in RLHF for LLMs
  8. 2024The N+ Implementation Details of RLHF with PPO: A Case Study on TL;DR Summarization