mlsanityData to 13 Oct 2026

H2O Open Ecosystem for State-of-the-art Large Language Models

arXiv 2310.13012 ↗First posted 17 Oct 2023cs.CL cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) represent a revolution in AI. However, they also pose many significant risks, such as the presence of biased, private, copyrighted or harmful text. For this reason we need open, transparent and safe solutions. We introduce a complete open-source ecosystem for developing and testing LLMs. The goal of this project is to boost open alternatives to closed-source approaches. We release h2oGPT, a family of fine-tuned LLMs of diverse sizes. We also introduce H2O LLM Studio, a framework and no-code GUI designed for efficient fine-tuning, evaluation, and deployment of LLMs using the most recent state-of-the-art techniques. Our code and models are fully open-source. We believe this work helps to boost AI development and make it more accessible, efficient and trustworthy. The demo is available at: https://gpt.h2o.ai/

Key facts. Cited by 2 papers in the map (data through 13 Oct 2026). Builds on 21 of its 31 references that are in the map. First posted on arXiv 17 Oct 2023.

Builds on

21 of 31 references.

References · 21

  1. 2017Attention Is All You Needcited by 60,106
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,304
  3. 2020Language Models are Few-Shot Learnerscited by 32,320
  4. 2022Training language models to follow instructions with human feedbackcited by 15,856
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,231
  6. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,072
  7. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,322
  8. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,884
  9. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,880
  10. 2021On the Opportunities and Risks of Foundation Modelscited by 3,999
  11. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,500
  12. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,388
9 more
  1. 2020Learning to summarize from human feedbackcited by 2,575
  2. 2019Fine-Tuning Language Models from Human Preferencescited by 2,139
  3. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,978
  4. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,120
  5. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 760
  6. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 731
  7. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 717
  8. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 711
  9. 2019FlauBERT: Unsupervised Language Model Pre-training for Frenchcited by 153

Built on by

Cited by year

Citing papers per year: 2024: 2.

2 papers.

  1. 2024LLMs with Industrial Lens: Deciphering the Challenges and Prospects -- A Surveycited by 12
  2. 2024No Size Fits All: The Perils and Pitfalls of Leveraging LLMs Vary with Company Sizecited by 3