# Fine-Tuning Qwen 2.5 3B for Realistic Movie Dialogue Generation

arXiv: https://arxiv.org/abs/2502.16274 · HTML: https://mlsanity.com/paper/2502.16274/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 34 references that are in the map.
- First posted on arXiv 22 Feb 2025.

## Builds on (most cited first, 4 of 4)

- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning](https://mlsanity.com/paper/2307.08691/) (2023)
- [NEFTune: Noisy Embeddings Improve Instruction Finetuning](https://mlsanity.com/paper/2310.05914/) (2023)

## Built on by (most cited first, 1 of 1)

- [Gradient-Based Model Fingerprinting for LLM Similarity Detection and Family Classification](https://mlsanity.com/paper/2506.01631/) (2025)

## Abstract (arXiv)

The Qwen 2.5 3B base model was fine-tuned to generate contextually rich and engaging movie dialogue, leveraging the Cornell Movie-Dialog Corpus, a curated dataset of movie conversations. Due to the limitations in GPU computing and VRAM, the training process began with the 0.5B model progressively scaling up to the 1.5B and 3B versions as efficiency improvements were implemented. The Qwen 2.5 series, developed by Alibaba Group, stands at the forefront of small open-source pre-trained models, particularly excelling in creative tasks compared to alternatives like Meta's Llama 3.2 and Google's Gemma. Results demonstrate the ability of small models to produce high-quality, realistic dialogue, offering a promising approach for real-time, context-sensitive conversation generation.