mlsanityData to 7 Oct 2026

Self-QA: Unsupervised Knowledge Guided Language Model Alignment

arXiv 2305.11952 ↗First posted 19 May 2023cs.CLcited by 8

On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods

Abstract (arXiv)

Large-scale language models like ChatGPT and GPT-4 have gained attention for their impressive conversational and generative capabilities. However, the creation of supervised paired question-answering data for instruction tuning presents formidable challenges. This endeavor necessitates substantial human effort for data annotation and wrestles with issues concerning data quality, diversity, accuracy, and other related factors. To overcome these obstacles, we introduce an innovative framework named Self-QA, which replaces the traditional practice of human-written instruction seeds with a vast amount of unsupervised knowledge, enabling the model to generate a larger quantity of correct and domain-specific instruction data. The effectiveness of our proposed method is demonstrated through experiments conducted on unsupervised corpora from various domains.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 6 of its 12 references that are in the map. First posted on arXiv 19 May 2023.

Builds on

6 of 12 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervision4 May
  2. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructions20 Dec
  3. 2022Training language models to follow instructions with human feedback4 Mar

Other references · 3

  1. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  2. 2023Instruction Tuning with GPT-4cited by 676
  3. 2022TranS: Transition-based Knowledge Graph Embedding with Synthetic Relation Representationcited by 7

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 2; 2025: 4.

8 papers.

  1. 2023Self-Alignment with Instruction Backtranslationcited by 149
  2. 2024Human-Instruction-Free LLM Self-Alignment with Limited Samplescited by 25
  3. 2023CFGPT: Chinese Financial Assistant with Large Language Modelcited by 11
  4. 2025GraphGen: Enhancing Supervised Fine-Tuning for LLMs with Knowledge-Driven Synthetic Data Generationcited by 7
  5. 2024ContraSolver: Self-Alignment of Language Models by Resolving Internal Preference Contradictionscited by 4
  6. 2025DeepThink: Aligning Language Models with Domain-Specific User Intentscited by 2
  7. 2025Not All Documents Are What You Need for Extracting Instruction Tuning Datacited by 1
  8. 2025QianfanHuijin Technical Report: A Novel Multi-Stage Training Paradigm for Finance Industrial LLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025CALM: Unleashing the Cross-Lingual Self-Aligning Ability of Language Model Question Answering
  2. 2025CYCLE-INSTRUCT: Fully Seed-Free Instruction Tuning via Dual Self-Training and Cycle Consistency
  3. 2024KBAlign: Efficient Self Adaptation on Specific Knowledge Bases
  4. 2024Don't Just Say "I don't know"! Self-aligning Large Language Models for Responding to Unknown Questions with Explanations
  5. 2024Rapidly Developing High-quality Instruction Data and Evaluation Benchmark for Large Language Models with Minimal Human Effort: A Case Study on Japanese
  6. 2023Aligning Instruction Tasks Unlocks Large Language Models as Zero-Shot Relation Extractors
  7. 2022Large Language Models Can Self-Improve
  8. 2024Rethinking Data Synthesis: A Teacher Model Training Recipe with Interpretation