Long Input Benchmark for Russian Analysis
Not on the map. Why?
Abstract (arXiv)
Recent advancements in Natural Language Processing (NLP) have fostered the development of Large Language Models (LLMs) that can solve an immense variety of tasks. One of the key aspects of their application is their ability to work with long text documents and to process long sequences of tokens. This has created a demand for proper evaluation of long-context understanding. To address this need for the Russian language, we propose LIBRA (Long Input Benchmark for Russian Analysis), which comprises 21 adapted datasets to study the LLM's abilities to understand long texts thoroughly. The tests are divided into four complexity groups and allow the evaluation of models across various context lengths ranging from 4k up to 128k tokens. We provide the open-source datasets, codebase, and public leaderboard for LIBRA to guide forthcoming research.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 20 of its 29 references that are in the map. First posted on arXiv 5 Aug 2024.
Builds on
20 of 29 references.
References · 20
- 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
- 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
- 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
- 2020Longformer: The Long-Document Transformercited by 3,362
- 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
- 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
- 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
- 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
- 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,259
- 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,085
- 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 475
- 2015Towards AI-Complete Question Answering: A Set of Prerequisite Toy Taskscited by 309
8 more
- 2023L-Eval: Instituting Standardized Evaluation for Long Context Language Modelscited by 232
- 2024BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystackcited by 211
- 2021QuALITY: Question Answering with Long Input Texts, Yes!cited by 204
- 2023ZeroSCROLLS: A Zero-Shot Benchmark for Long Text Understandingcited by 175
- 2023Unlimiformer: Long-Range Transformers with Unlimited Length Inputcited by 127
- 2020RussianSuperGLUE: A Russian Language Understanding Evaluation Benchmarkcited by 53
- 2016Towards Machine Comprehension of Spoken Content: Initial TOEFL Listening Comprehension Test by Machinecited by 23
- 2022TAPE: Assessing Few-shot Russian Language Understandingcited by 11
Built on by
Citing papers per year: 2025: 1.
1 paper.