# Baize: An Open-Source Chat Model with Parameter-Efficient Tuning on Self-Chat Data

arXiv: https://arxiv.org/abs/2304.01196 · HTML: https://mlsanity.com/paper/2304.01196/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Apr 2023.

## Built on by (most cited first, 13 of 13)

- [GeoGalactica: A Scientific Large Language Model in Geoscience](https://mlsanity.com/paper/2401.00434/) (2023)
- [Aligning Large Language Models with Human: A Survey](https://mlsanity.com/paper/2307.12966/) (2023)
- [Interactive Natural Language Processing](https://mlsanity.com/paper/2305.13246/) (2023)
- [Under the Surface: Tracking the Artifactuality of LLM-Generated Data](https://mlsanity.com/paper/2401.14698/) (2024)
- [TOOLVERIFIER: Generalization to New Tools via Self-Verification](https://mlsanity.com/paper/2402.14158/) (2024)
- [GRPO-λ: Credit Assignment improves LLM Reasoning](https://mlsanity.com/paper/2510.00194/) (2025)
- [Learning from Committee: Reasoning Distillation from a Mixture of Teachers with Peer-Review](https://mlsanity.com/paper/2410.03663/) (2024)
- [Towards Infinite-Long Prefix in Transformer](https://mlsanity.com/paper/2406.14036/) (2024)
- [EvoLlama: Enhancing LLMs' Understanding of Proteins via Multimodal Structure and Sequence Representations](https://mlsanity.com/paper/2412.11618/) (2024)
- [LaRS: Latent Reasoning Skills for Chain-of-Thought Reasoning](https://mlsanity.com/paper/2312.04684/) (2023)
- [Small Language Model as Data Prospector for Large Language Model](https://mlsanity.com/paper/2412.09990/) (2024)
- [FABRIC: Framework for Agent-Based Realistic Intelligence Creation](https://mlsanity.com/paper/2510.17995/) (2025)
- [Scaling-Aware Adapter for Structure-Grounded LLM Reasoning](https://mlsanity.com/paper/2602.02780/) (2026)

## Abstract (arXiv)

Chat models, such as ChatGPT, have shown impressive capabilities and have been rapidly adopted across numerous domains. However, these models are only accessible through a restricted API, creating barriers for new research and progress in the field. We propose a pipeline that can automatically generate a high-quality multi-turn chat corpus by leveraging ChatGPT to engage in a conversation with itself. Subsequently, we employ parameter-efficient tuning to enhance LLaMA, an open-source large language model. The resulting model, named Baize, demonstrates good performance in multi-turn dialogues with guardrails that minimize potential risks. Furthermore, we propose a new technique called Self-Distill with Feedback, to further improve the performance of the Baize models with feedback from ChatGPT. The Baize models and data are released for research purposes only at https://github.com/project-baize/baize-chatbot. An online demo is also available at https://huggingface.co/spaces/project-baize/chat-with-baize.