# PACuna: Automated Fine-Tuning of Language Models for Particle Accelerators

arXiv: https://arxiv.org/abs/2310.19106 · HTML: https://mlsanity.com/paper/2310.19106/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 16 references that are in the map.
- First posted on arXiv 29 Oct 2023.

## Builds on (most cited first, 7 of 7)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Neural Text Generation with Unlikelihood Training](https://mlsanity.com/paper/1908.04319/) (2019)
- [VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Tasks](https://mlsanity.com/paper/2305.11175/) (2023)
- [PointLLM: Empowering Large Language Models to Understand Point Clouds](https://mlsanity.com/paper/2308.16911/) (2023)
- [LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models](https://mlsanity.com/paper/2309.12307/) (2023)
- [A Practical Toolkit for Multilingual Question and Answer Generation](https://mlsanity.com/paper/2305.17416/) (2023)

## Built on by (most cited first, 3 of 3)

- [Large Language Models for Human-Machine Collaborative Particle Accelerator Tuning through Natural Language](https://mlsanity.com/paper/2405.08888/) (2024)
- [PhysBERT: A Text Embedding Model for Physics Scientific Literature](https://mlsanity.com/paper/2408.09574/) (2024)
- [Towards Unlocking Insights from Logbooks Using AI](https://mlsanity.com/paper/2406.12881/) (2024)

## Abstract (arXiv)

Navigating the landscape of particle accelerators has become increasingly challenging with recent surges in contributions. These intricate devices challenge comprehension, even within individual facilities. To address this, we introduce PACuna, a fine-tuned language model refined through publicly available accelerator resources like conferences, pre-prints, and books. We automated data collection and question generation to minimize expert involvement and make the data publicly available. PACuna demonstrates proficiency in addressing intricate accelerator questions, validated by experts. Our approach shows adapting language models to scientific domains by fine-tuning technical texts and auto-generated corpora capturing the latest developments can further produce pre-trained models to answer some intricate questions that commercially available assistants cannot and can serve as intelligent assistants for individual facilities.