mlsanityData to 14 Oct 2026

Reflections from the 2024 Large Language Model (LLM) Hackathon for Applications in Materials Science and Chemistry

arXiv 2411.15221 ↗First posted 20 Nov 2024cs.LG cond-mat.mtrl-sci physics.chem-phcited by 8

Not on the map. Why?

Abstract (arXiv)

Here, we present the outcomes from the second Large Language Model (LLM) Hackathon for Applications in Materials Science and Chemistry, which engaged participants across global hybrid locations, resulting in 34 team submissions. The submissions spanned seven key application areas and demonstrated the diverse utility of LLMs for applications in (1) molecular and material property prediction; (2) molecular and material design; (3) automation and novel interfaces; (4) scientific communication and education; (5) research data management and automation; (6) hypothesis generation and evaluation; and (7) knowledge extraction and reasoning from scientific literature. Each team submission is presented in a summary table with links to the code and as brief papers in the appendix. Beyond team results, we discuss the hackathon event and its hybrid format, which included physical hubs in Toronto, Montreal, San Francisco, Berlin, Lausanne, and Tokyo, alongside a global online hub to enable local and virtual collaboration. Overall, the event highlighted significant improvements in LLM capabilities since the previous year's hackathon, suggesting continued expansion of LLMs for applications in materials science and chemistry research. These outcomes demonstrate the dual utility of LLMs as both multipurpose models for diverse machine learning tasks and platforms for rapid prototyping custom applications in scientific research.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 55 of its 247 references that are in the map. First posted on arXiv 20 Nov 2024.

Builds on

55 of 247 references.

References · 55

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  6. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  7. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  8. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  9. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  10. 2023Mistral 7Bcited by 3,164
  11. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  12. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
38 more
  1. 2024Mixtral of Expertscited by 1,823
  2. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 832
  3. 2022A Survey on In-context Learningcited by 721
  4. 2023ChemCrow: Augmenting large-language models with chemistry toolscited by 631
  5. 2024TinyLlama: An Open-Source Small Language Modelcited by 618
  6. 2020ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Predictioncited by 337
  7. 2024Many-Shot In-Context Learningcited by 206
  8. 2024Large Language Models for Education: A Survey and Outlookcited by 188
  9. 2023What can Large Language Models do in chemistry? A comprehensive benchmark on eight taskscited by 185
  10. 2023Large Language Models for Generative Information Extraction: A Surveycited by 185
  11. 2023RestGPT: Connecting Large Language Models with Real-World RESTful APIscited by 135
  12. 2024LlaSMol: Advancing Large Language Models for Chemistry with a Large-Scale, Comprehensive, High-Quality Instruction Tuning Datasetcited by 128
  13. 2024Guiding LLMs The Right Way: Fast, Non-Invasive Constrained Generationcited by 101
  14. 2023Large Language Models for Automated Open-domain Scientific Hypotheses Discoverycited by 100
  15. 2024Reducing hallucination in structured outputs via Retrieval-Augmented Generationcited by 93
  16. 2023Cognitive Mirage: A Review of Hallucinations in Large Language Modelscited by 84
  17. 2024Hypothesis Generation with Large Language Modelscited by 71
  18. 2024SciAgent: Tool-augmented Language Models for Scientific Reasoningcited by 70
  19. 2024ORGANA: A Robotic Assistant for Automated Chemistry Experimentation and Characterizationcited by 61
  20. 2024HoneyComb: A Flexible LLM-Based Agent System for Materials Sciencecited by 57
  21. 2024A Sober Look at LLMs for Material Discovery: Are They Actually Good for Bayesian Optimization Over Molecules?cited by 48
  22. 2023Can Large Language Models Empower Molecular Property Prediction?cited by 44
  23. 2024LLMatDesign: Autonomous Materials Discovery with Large Language Modelscited by 42
  24. 2024Are LLMs Ready for Real-World Materials Discovery?cited by 40
  25. 2024IsoBench: Benchmarking Multimodal Foundation Models on Isomorphic Representationscited by 38
  26. 2024Are large language models superhuman chemists?cited by 34
  27. 2023ChatGPT-powered Conversational Drug Editing Using Retrieval and Domain Feedbackcited by 31
  28. 2023LLM-Prop: Predicting Physical And Electronic Properties Of Crystalline Solids From Their Text Descriptionscited by 29
  29. 2024Multimodal Large Language Models for Inverse Molecular Design with Retrosynthetic Planningcited by 29
  30. 2023Multimodal Foundation Models for Material Property Prediction and Discoverycited by 19
  31. 2023Agent-based Learning of Materials Datasets from Scientific Literaturecited by 15
  32. 2024Are Large Language Models (LLMs) Good Social Predictors?cited by 13
  33. 2024Large Language Models for Scientific Information Extraction: An Empirical Study for Virologycited by 12
  34. 2023Harnessing the Power of Adversarial Prompting and Large Language Models for Robust Hypothesis Generation in Astronomycited by 11
  35. 2023MaScQA: A Question Answering Dataset for Investigating Materials Science Knowledge of Large Language Modelscited by 11
  36. 2024Scientific Hypothesis Generation by a Large Language Model: Laboratory Validation in Breast Cancer Treatmentcited by 11
  37. 2024Less can be more for predicting properties with large language modelscited by 11
  38. 2024Regression with Large Language Models for Materials and Molecular Property Predictioncited by 11

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 5; 2026: 2.

8 papers.

  1. 2025DREAMS: Density Functional Theory Based Research Engine for Agentic Materials Simulationcited by 29
  2. 2024Foundational Large Language Models for Materials Researchcited by 18
  3. 2025Chemical reasoning in LLMs unlocks strategy-aware synthesis planning and reaction mechanism elucidationcited by 17
  4. 2025Synthelite: Chemist-aligned and feasibility-aware synthesis planning with LLMscited by 10
  5. 2025SEAM: Semantically Equivalent Across Modalities Benchmark for Vision-Language Modelscited by 6
  6. 2025ComProScanner: A multi-agent based framework for composition-property structured data extraction from scientific literaturecited by 1
  7. 2026Autonomous Multi-Agent AI for High-Throughput Polymer Informatics: From Property Prediction to Generative Design Across Synthetic and Bio-Polymerscited by 1
  8. 2026From Knowledge to Action: Outcomes of the 2025 Large Language Model (LLM) Hackathon for Applications in Materials Science and Chemistrycited by 1