Reactor Mk.1 performances: MMLU, HumanEval and BBH test results
Not on the map. Why?
Abstract (arXiv)
The paper presents the performance results of Reactor Mk.1, ARCs flagship large language model, through a benchmarking process analysis. The model utilizes the Lychee AI engine and possesses less than 100 billion parameters, resulting in a combination of efficiency and potency. The Reactor Mk.1 outperformed models such as GPT-4o, Claude Opus, and Llama 3, with achieved scores of 92% on the MMLU dataset, 91% on HumanEval dataset, and 88% on BBH dataset. It excels in both managing difficult jobs and reasoning, establishing as a prominent AI solution in the present cutting-edge AI technology.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 4 of its 11 references that are in the map. First posted on arXiv 15 Jun 2024.
Builds on
4 of 11 references.
References · 4
- 2021Evaluating Large Language Models Trained on Codecited by 8,320
- 2020Measuring Massive Multitask Language Understandingcited by 7,963
- 2024Mixtral of Expertscited by 1,823
- 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
Built on by
Citing papers per year: 2024: 1.
1 paper.