# Sakura at BEA 2026 Shared Task 1: What Makes Vocabulary Difficult?

arXiv: https://arxiv.org/abs/2605.14257 · HTML: https://mlsanity.com/paper/2605.14257/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 35 references that are in the map.
- First posted on arXiv 14 May 2026.

## Builds on (most cited first, 14 of 14)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools](https://mlsanity.com/paper/2406.12793/) (2024)
- [mmBERT: A Modern Multilingual Encoder with Annealed Language Learning](https://mlsanity.com/paper/2509.06888/) (2025)
- [SemEval-2021 Task 1: Lexical Complexity Prediction](https://mlsanity.com/paper/2106.00473/) (2021)
- [UniHD at TSAR-2022 Shared Task: Is Compute All We Need for Lexical Simplification?](https://mlsanity.com/paper/2301.01764/) (2023)
- [Japanese Lexical Complexity for Non-Native Readers: A New Dataset](https://mlsanity.com/paper/2306.17399/) (2023)
- [Beyond Film Subtitles: Is YouTube the Best Approximation of Spoken Vocabulary?](https://mlsanity.com/paper/2410.03240/) (2024)
- [Difficult for Whom? A Study of Japanese Lexical Complexity](https://mlsanity.com/paper/2410.18567/) (2024)
- [Investigating Large Language Models for Complex Word Identification in Multilingual and Multidomain Setups](https://mlsanity.com/paper/2411.01706/) (2024)

## Abstract (arXiv)

We describe two types of models for vocabulary difficulty prediction: a high-accuracy black-box model, which achieved the top shared task result in the open track, and an explainable model, which outperforms a fine-tuned encoder baseline. As the black-box model, we fine-tuned an LLM using a soft-target loss function for effective application to the rating task, achieving r > 0.91. The explainable model provides insights into what impacts the difficulty of each item while maintaining a strong correlation (r > 0.77). We further analyze the results, demonstrating that the difficulty of items in the British Council's Knowledge-based Vocabulary Lists (KVL) is often affected by spelling difficulty or the construction of the test items, in addition to the genuine production difficulty of the words. We make our code available online at https://github.com/ynklab/vocabulary-difficulty .