mlsanityData to 14 Oct 2026

Reactor Mk.1 performances: MMLU, HumanEval and BBH test results

arXiv 2406.10515 ↗First posted 15 Jun 2024cs.AI cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

The paper presents the performance results of Reactor Mk.1, ARCs flagship large language model, through a benchmarking process analysis. The model utilizes the Lychee AI engine and possesses less than 100 billion parameters, resulting in a combination of efficiency and potency. The Reactor Mk.1 outperformed models such as GPT-4o, Claude Opus, and Llama 3, with achieved scores of 92% on the MMLU dataset, 91% on HumanEval dataset, and 88% on BBH dataset. It excels in both managing difficult jobs and reasoning, establishing as a prominent AI solution in the present cutting-edge AI technology.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 4 of its 11 references that are in the map. First posted on arXiv 15 Jun 2024.

Builds on

4 of 11 references.

References · 4

  1. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  3. 2024Mixtral of Expertscited by 1,823
  4. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.