# CLASS-IT: Conversational and Lecture-Aligned Small-Scale Instruction Tuning for BabyLMs

arXiv: https://arxiv.org/abs/2510.25364 · HTML: https://mlsanity.com/paper/2510.25364/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 25 references that are in the map.
- First posted on arXiv 29 Oct 2025.

## Builds on (most cited first, 6 of 6)

- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [BLiMP: The Benchmark of Linguistic Minimal Pairs for English](https://mlsanity.com/paper/1912.00582/) (2019)
- [Are Emergent Abilities of Large Language Models a Mirage?](https://mlsanity.com/paper/2304.15004/) (2023)
- [Why Does Surprisal From Larger Transformer-Based Language Models Provide a Poorer Fit to Human Reading Times?](https://mlsanity.com/paper/2212.12131/) (2022)
- [Baby's CoThought: Leveraging Large Language Models for Enhanced Reasoning in Compact Models](https://mlsanity.com/paper/2308.01684/) (2023)
- [BabyStories: Can Reinforcement Learning Teach Baby Language Models to Write Better Stories?](https://mlsanity.com/paper/2310.16681/) (2023)

## Abstract (arXiv)

This work investigates whether small-scale LMs can benefit from instruction tuning. We compare conversational and question-answering instruction tuning datasets, applied either in a merged or sequential curriculum, using decoder-only models with 100M and 140M parameters. Evaluation spans both fine-tuning (SuperGLUE) and zero-shot (BLiMP, EWoK, WUGs, entity tracking, and psycholinguistic correlation) settings. Results show that instruction tuning yields small but consistent gains in fine-tuning scenarios, with sequential curricula outperforming merged data; however, improvements do not consistently transfer to zero-shot tasks, suggesting a trade-off between interaction-focused adaptation and broad linguistic generalization. These results highlight both the potential and the constraints of adapting human-inspired learning strategies to low-resource LMs, and point toward hybrid, curriculum-based approaches for enhancing generalization under ecological training limits.