mlsanityData to 7 Oct 2026

Quokka: An Open-source Large Language Model ChatBot for Material Science

arXiv 2401.01089 ↗First posted 2 Jan 2024cs.CL cs.AI cs.CEcited by 1

Not on the map. Why?

Abstract (arXiv)

This paper presents the development of a specialized chatbot for materials science, leveraging the Llama-2 language model, and continuing pre-training on the expansive research articles in the materials science domain from the S2ORC dataset. The methodology involves an initial pretraining phase on over one million domain-specific papers, followed by an instruction-tuning process to refine the chatbot's capabilities. The chatbot is designed to assist researchers, educators, and students by providing instant, context-aware responses to queries in the field of materials science. We make the four trained checkpoints (7B, 13B, with or without chat ability) freely available to the research community at https://github.com/Xianjun-Yang/Quokka.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 22 of its 30 references that are in the map. First posted on arXiv 2 Jan 2024.

Builds on

22 of 30 references.

References · 22

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  6. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  7. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  8. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
  9. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 755
  10. 2023MedAlpaca -- An Open-Source Collection of Medical Conversational AI Models and Training Datacited by 282
  11. 2023Mol-Instructions: A Large-Scale Biomolecular Instruction Dataset for Large Language Modelscited by 131
  12. 2023Exploring the Limits of ChatGPT for Query or Aspect-based Text Summarizationcited by 122
10 more
  1. 2023AlpaCare:Instruction-tuned Large Language Models for Medical Applicationcited by 88
  2. 2023Lawyer LLaMA Technical Reportcited by 54
  3. 2023MatSci-NLP: Evaluating Scientific Language Models on Materials Science Language Tasks Using Text-to-Schema Modelingcited by 35
  4. 2020The SOFC-Exp Corpus and Neural Approaches to Information Extraction in the Materials Science Domaincited by 32
  5. 2023HoneyBee: Progressive Instruction Finetuning of Large Language Models for Materials Sciencecited by 22
  6. 2023DoG-Instruct: Towards Premium Instruction-Tuning Data via Text-Grounded Instruction Wrappingcited by 16
  7. 2023Text Alignment Is An Efficient Unified Model for Massive NLP Taskscited by 8
  8. 2023M2Hub: Unlocking the Potential of Machine Learning for Materials Discoverycited by 8
  9. 2022PcMSP: A Dataset for Scientific Action Graphs Extraction from Polycrystalline Materials Synthesis Procedure Textcited by 6
  10. 2023MatKB: Semantic Search for Polycrystalline Materials Synthesis Procedurescited by 2

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024PLLaMa: An Open-source Large Language Model for Plant Sciencecited by 22