# RakutenAI-7B: Extending Large Language Models for Japanese

arXiv: https://arxiv.org/abs/2403.15484 · HTML: https://mlsanity.com/paper/2403.15484/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 17 references that are in the map.
- First posted on arXiv 21 Mar 2024.

## Builds on (most cited first, 10 of 10)

- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge](https://mlsanity.com/paper/1811.00937/) (2018)
- [Language Models are Multilingual Chain-of-Thought Reasoners](https://mlsanity.com/paper/2210.03057/) (2022)
- [Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus](https://mlsanity.com/paper/2104.08758/) (2021)
- [XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languages](https://mlsanity.com/paper/2106.13822/) (2021)
- [The Multilingual Amazon Reviews Corpus](https://mlsanity.com/paper/2010.02573/) (2020)
- [ROUGE 2.0: Updated and Improved Measures for Evaluation of Summarization Tasks](https://mlsanity.com/paper/1803.01937/) (2018)

## Built on by (most cited first, 13 of 13)

- [Fishing for Magikarp: Automatically Detecting Under-trained Tokens in Large Language Models](https://mlsanity.com/paper/2405.05417/) (2024)
- [Exploring Design Choices for Building Language-Specific LLMs](https://mlsanity.com/paper/2406.14670/) (2024)
- [Meltemi: The first open Large Language Model for Greek](https://mlsanity.com/paper/2407.20743/) (2024)
- [Adapting LLMs to Hebrew: Unveiling DictaLM 2.0 with Enhanced Vocabulary and Instruction Capabilities](https://mlsanity.com/paper/2407.07080/) (2024)
- [SUTRA: Scalable Multilingual Language Model Architecture](https://mlsanity.com/paper/2405.06694/) (2024)
- [TituLLMs: A Family of Bangla LLMs with Comprehensive Benchmarking](https://mlsanity.com/paper/2502.11187/) (2025)
- [Continually Adding New Languages to Multilingual Language Models](https://mlsanity.com/paper/2509.11414/) (2025)
- [A Dual-Layered Evaluation of Geopolitical and Cultural Bias in LLMs](https://mlsanity.com/paper/2506.21881/) (2025)
- [Why We Build Local Large Language Models: An Observational Analysis from 35 Japanese and Multilingual LLMs](https://mlsanity.com/paper/2412.14471/) (2024)
- [XL-SafetyBench: A Country-Grounded Cross-Cultural Benchmark for LLM Safety and Cultural Sensitivity](https://mlsanity.com/paper/2605.05662/) (2026)
- [SnakModel: Lessons Learned from Training an Open Danish Large Language Model](https://mlsanity.com/paper/2412.12956/) (2024)
- [Exploring Gender Bias Beyond Occupational Titles](https://mlsanity.com/paper/2507.02679/) (2025)
- [A Locally Deployed RAG-Based Academic Advising System for Course Selection](https://mlsanity.com/paper/2606.02983/) (2026)

## Abstract (arXiv)

We introduce RakutenAI-7B, a suite of Japanese-oriented large language models that achieve the best performance on the Japanese LM Harness benchmarks among the open 7B models. Along with the foundation model, we release instruction- and chat-tuned models, RakutenAI-7B-instruct and RakutenAI-7B-chat respectively, under the Apache 2.0 license.