mlsanityData to 14 Oct 2026

Open Artificial Knowledge

arXiv 2407.14371 ↗First posted 19 Jul 2024cs.CL cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

The tremendous success of chat-based AI systems like ChatGPT, Claude, and Gemini stems from Large Language Models (LLMs) trained on vast amount of datasets. However, acquiring high-quality, diverse, and ethically sourced training data remains a significant challenge. We introduce the Open Artificial Knowledge (OAK) dataset, a large-scale resource of over 500 million tokens (at the moment of writing) designed to address this issue. OAK leverages an ensemble of state-of-the-art LLMs, including GPT4o, LLaMa3-70B, LLaMa3-8B, Mixtral-8x7B, Gemma-7B, and Gemma-2-9B , to generate high-quality text across diverse domains, guided by Wikipedia's main categories. Our methodology ensures broad knowledge coverage while maintaining coherence and factual accuracy. The OAK dataset aims to foster the development of more capable and aligned language models while addressing critical issues of data scarcity and privacy in LLM training, and it is freely available on www.oakdataset.org.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 25 of its 35 references that are in the map. First posted on arXiv 19 Jul 2024.

Builds on

25 of 35 references.

References · 25

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  5. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  6. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  7. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,045
  8. 2024Yi: Open Foundation Models by 01.AIcited by 719
  9. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 606
  10. 2023Textbooks Are All You Need II: phi-1.5 technical reportcited by 565
  11. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 557
  12. 2022Language Models are Realistic Tabular Data Generatorscited by 294
13 more
  1. 2024On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Surveycited by 244
  2. 2023Large Language Model Alignment: A Surveycited by 243
  3. 2024Long-form factuality in large language modelscited by 155
  4. 2024Best Practices and Lessons Learned on Synthetic Datacited by 106
  5. 2023Synthetic Data Generation with Large Language Models for Text Classification: Potential and Limitationscited by 100
  6. 2023Exploiting Asymmetry for Synthetic Training Data Generation: SynthIE and the Case of Information Extractioncited by 89
  7. 2023FactKB: Generalizable Factuality Evaluation using Language Models Enhanced with Factual Knowledgecited by 55
  8. 2023Prompt Engineering a Prompt Engineercited by 53
  9. 2024Efficient Prompting Methods for Large Language Models: A Surveycited by 47
  10. 2024Prompt Design and Engineering: Introduction and Advanced Methodscited by 45
  11. 2023Deep Learning for Code Intelligence: Survey, Benchmark and Toolkitcited by 22
  12. 2024Synthetic Data in AI: Challenges, Applications, and Ethical Implicationscited by 21
  13. 2024Refined Direct Preference Optimization with Synthetic Data for Behavioral Alignment of LLMscited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Multilingual Multi-Label Emotion Classification at Scale with Synthetic Datacited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026It's All Training: A Fully Synthetic Single-Stage Recipe for LLMs
  2. 2025Challenges and Applications of Large Language Models: A Comparison of GPT and DeepSeek family of models
  3. 2024Under the Surface: Tracking the Artifactuality of LLM-Generated Data
  4. 2023Unifying Large Language Models and Knowledge Graphs: A Roadmap
  5. 2026State of AI: An Empirical 100 Trillion Token Study with OpenRouter
  6. 2023ChatGPT's One-year Anniversary: Are Open-Source Large Language Models Catching up?
  7. 2025WILDCHAT-50M: A Deep Dive Into the Role of Synthetic Data in Post-Training
  8. 2023Large Knowledge Model: Perspectives and Challenges