mlsanityData to 7 Oct 2026

Condor: Enhance LLM Alignment with Knowledge-Driven Data Synthesis and Refinement

arXiv 2501.12273 ↗First posted 21 Jan 2025cs.CL cs.AIcited by 17

On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods

also in 1 more
Abstract (arXiv)

The quality of Supervised Fine-Tuning (SFT) data plays a critical role in enhancing the conversational capabilities of Large Language Models (LLMs). However, as LLMs become more advanced, the availability of high-quality human-annotated SFT data has become a significant bottleneck, necessitating a greater reliance on synthetic training data. In this work, we introduce Condor, a novel two-stage synthetic data generation framework that incorporates World Knowledge Tree and Self-Reflection Refinement to produce high-quality SFT data at scale. Our experimental results demonstrate that a base model fine-tuned on only 20K Condor-generated samples achieves superior performance compared to counterparts. The additional refinement stage in Condor further enables iterative self-improvement for LLMs at various scales (up to 72B), validating the effectiveness of our approach. Furthermore, our investigation into the scaling for synthetic data in post-training reveals substantial unexplored potential for performance improvements, opening promising avenues for future research.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 33 of its 47 references that are in the map. First posted on arXiv 21 Jan 2025.

Builds on

33 of 47 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing12 Jun
  2. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructions20 Dec

Other references · 31

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  7. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  8. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  9. 2021Program Synthesis with Large Language Modelscited by 3,509
  10. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  11. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  12. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
19 more
  1. 2024Qwen2 Technical Reportcited by 2,076
  2. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  3. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,093
  4. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 883
  5. 2024Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluatorscited by 865
  6. 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 851
  7. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 825
  8. 2022Large Language Models Can Self-Improvecited by 656
  9. 2024Self-Rewarding Language Modelscited by 654
  10. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 486
  11. 2024From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipelinecited by 484
  12. 2024MT-Bench-101: A Fine-Grained Benchmark for Evaluating Large Language Models in Multi-Turn Dialoguescited by 247
  13. 2024WildBench: Benchmarking LLMs with Challenging Tasks from Real Users in the Wildcited by 211
  14. 20233D-VisTA: Pre-trained Transformer for 3D Vision and Text Alignmentcited by 201
  15. 2023Evaluating the Performance of Large Language Models on GAOKAO Benchmarkcited by 182
  16. 2024FOFO: A Benchmark to Evaluate LLMs' Format-Following Capabilitycited by 79
  17. 2024CompassJudger-1: All-in-one Judge Model Helps Model Evaluation and Evolutioncited by 42
  18. 2024Rethinking Data Selection for Supervised Fine-Tuningcited by 35
  19. 2024I-SHEEP: Self-Alignment of LLM from Scratch through an Iterative Self-Enhancement Paradigmcited by 27

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 11; 2026: 6.

17 papers.

Sort:

By area:

  1. 2025Intern-S1: A Scientific Multimodal Foundation Modelcited by 49
  2. 2025OmniAlign-V: Towards Enhanced Alignment of MLLMs with Human Preferencecited by 27
  3. 2025A Strategic Coordination Framework of Small LLMs Matches Large LLMs in Data Synthesiscited by 9
  4. 2025A Survey on Efficient Large Language Model Training: From Data-centric Perspectivescited by 9
  5. 2025Middo: Model-Informed Dynamic Data Optimization for Enhanced LLM Fine-Tuning via Closed-Loop Learningcited by 7
  6. 2026Optimsyn: Influence-Guided Rubrics Optimization for Synthetic Data Generationcited by 4
  7. 2025GenieBlue: Integrating both Linguistic and Multimodal Capabilities for Large Language Models on Mobile Devicescited by 3
  8. 2025Bohdi: Heterogeneous LLM Fusion with Automatic Data Explorationcited by 3
  9. 2026Supervised Fine-Tuning versus Reinforcement Learning: A Study of Post-Training Methods for Large Language Modelscited by 3
  10. 2025MASTER: Enhancing Large Language Model via Multi-Agent Simulated Teachingcited by 2
  11. 2025DecIF: Improving Instruction-Following through Meta-Decompositioncited by 1
  12. 2025AAPA: Adversarially Anchored Preference Alignment for Post-Training of Large Language Modelscited by 1
5 more
  1. 2025GraphIF: Enhancing Multi-Turn Instruction Following for Large Language Models with Relation Graph Promptcited by 1
  2. 2026WIST: Web-Grounded Iterative Self-Play Tree for Domain-Targeted Reasoning Improvementcited by 0
  3. 2026HSS-Synth: Humanities and Social Sciences Data Synthesis for LLMscited by 0
  4. 2026Quantifying Error Tolerance in Synthetic Data: An Atomic-level Operand vs. Operator Perturbation Studycited by 0
  5. 2026SynCo: Data Synthesis Co-Training for Self-Evolving LLMs via Multi-Agent Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Massive Supervised Fine-tuning Experiments Reveal How Data, Layer, and Training Factors Shape LLM Alignment Quality
  2. 2024NILE: Internal Consistency Alignment in Large Language Models
  3. 2025Modular Techniques for Synthetic Long-Context Data Generation in Language Model Training and Evaluation
  4. 2025SyGra: A Unified Graph-Based Framework for Scalable Generation, Quality Tagging, and Management of Synthetic Data
  5. 2024A Survey on Data Synthesis and Augmentation for Large Language Models
  6. 2025GraphGen: Enhancing Supervised Fine-Tuning for LLMs with Knowledge-Driven Synthetic Data Generation
  7. 2024ToolFlow: Boosting LLM Tool-Calling Through Natural and Coherent Dialogue Synthesis
  8. 2025Refine-n-Judge: Curating High-Quality Preference Chains for LLM-Fine-Tuning