LokiLM: Technical Report
Not on the map. Why?
Abstract (arXiv)
In this work, we introduce LokiLM, a 1.4B parameter large language model trained on 500B tokens. Our model performs strongly in natural language reasoning tasks and achieves state-of-the-art performance among models with 1.5B parameters or less. LokiLM is trained using multi-teacher knowledge distillation and high-quality training data to achieve benchmark results competitive with larger models trained on significantly more tokens. We support these findings by introducing steps to avoid benchmark contamination and overfitting throughout our development process. Despite its promising performance, LokiLM exhibits a concerning amount of hallucinations and scores poorly on the TruthfulQA benchmark, so we do not release the model publicly.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). First posted on arXiv 10 Jul 2024.
Builds on
0 of 0 references.
No references here yet.
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Towards Reasoning Ability of Small Language Models
- 2025Effective Learning for Small Reasoning Models: An Empirical Study on 0.5B Reasoning LLMs
- 2023NPHardEval: Dynamic Benchmark on Reasoning Ability of Large Language Models via Complexity Classes
- 2023FELM: Benchmarking Factuality Evaluation of Large Language Models
- 2026Maistros: A Greek Large Language Model Adapted Through Knowledge Distillation From Large Reasoning Models
- 2025ReasoningWeekly: A General Knowledge and Verbal Reasoning Challenge for Large Language Models
- 2025MindVL: Towards Efficient and Effective Training of Multimodal Large Language Models on Ascend NPUs
- 2025MobileLLM-R1: Exploring the Limits of Sub-Billion Language Model Reasoners with Open Training Recipes