# Self-QA: Unsupervised Knowledge Guided Language Model Alignment

arXiv: https://arxiv.org/abs/2305.11952 · HTML: https://mlsanity.com/paper/2305.11952/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 6 of its 12 references that are in the map.
- First posted on arXiv 19 May 2023.

## Placed in

- LLM alignment / RLHF / preference optimisation / [LLM preference optimization methods](https://mlsanity.com/alignment/llm-preference-optimization-methods/)

## Builds on (most cited first, 6 of 6)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [Instruction Tuning with GPT-4](https://mlsanity.com/paper/2304.03277/) (2023)
- [Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervision](https://mlsanity.com/paper/2305.03047/) (2023)
- [TranS: Transition-based Knowledge Graph Embedding with Synthetic Relation Representation](https://mlsanity.com/paper/2204.08401/) (2022)

## Built on by (most cited first, 9 of 9)

- [Self-Alignment with Instruction Backtranslation](https://mlsanity.com/paper/2308.06259/) (2023)
- [Human-Instruction-Free LLM Self-Alignment with Limited Samples](https://mlsanity.com/paper/2401.06785/) (2024)
- [CFGPT: Chinese Financial Assistant with Large Language Model](https://mlsanity.com/paper/2309.10654/) (2023)
- [GraphGen: Enhancing Supervised Fine-Tuning for LLMs with Knowledge-Driven Synthetic Data Generation](https://mlsanity.com/paper/2505.20416/) (2025)
- [ContraSolver: Self-Alignment of Language Models by Resolving Internal Preference Contradictions](https://mlsanity.com/paper/2406.08842/) (2024)
- [DeepThink: Aligning Language Models with Domain-Specific User Intents](https://mlsanity.com/paper/2502.05497/) (2025)
- [Not All Documents Are What You Need for Extracting Instruction Tuning Data](https://mlsanity.com/paper/2505.12250/) (2025)
- [QianfanHuijin Technical Report: A Novel Multi-Stage Training Paradigm for Finance Industrial LLMs](https://mlsanity.com/paper/2512.24314/) (2025)
- [SAGE: Semantic Anchor-Guided Evolution for Grounded Medical QA Data Synthesis](https://mlsanity.com/paper/2610.08093/) (2026)

## Abstract (arXiv)

Large-scale language models like ChatGPT and GPT-4 have gained attention for their impressive conversational and generative capabilities. However, the creation of supervised paired question-answering data for instruction tuning presents formidable challenges. This endeavor necessitates substantial human effort for data annotation and wrestles with issues concerning data quality, diversity, accuracy, and other related factors. To overcome these obstacles, we introduce an innovative framework named Self-QA, which replaces the traditional practice of human-written instruction seeds with a vast amount of unsupervised knowledge, enabling the model to generate a larger quantity of correct and domain-specific instruction data. The effectiveness of our proposed method is demonstrated through experiments conducted on unsupervised corpora from various domains.