# Improving the Efficiency of Long Document Classification using Sentence Ranking Approach

arXiv: https://arxiv.org/abs/2506.07248 · HTML: https://mlsanity.com/paper/2506.07248/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 33 references that are in the map.
- First posted on arXiv 8 Jun 2025.

## Builds on (most cited first, 17 of 17)

- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Big Bird: Transformers for Longer Sequences](https://mlsanity.com/paper/2007.14062/) (2020)
- [Long Range Arena: A Benchmark for Efficient Transformers](https://mlsanity.com/paper/2011.04006/) (2020)
- [How to Fine-Tune BERT for Text Classification?](https://mlsanity.com/paper/1905.05583/) (2019)
- [Q8BERT: Quantized 8Bit BERT](https://mlsanity.com/paper/1910.06188/) (2019)
- [L3Cube-MahaCorpus and MahaBERT: Marathi Monolingual Corpus, Marathi BERT Language Models, and Resources](https://mlsanity.com/paper/2202.01159/) (2022)
- [Sparse and Continuous Attention Mechanisms](https://mlsanity.com/paper/2006.07214/) (2020)
- [Efficient Classification of Long Documents Using Transformers](https://mlsanity.com/paper/2203.11258/) (2022)
- [Indic-Transformers: An Analysis of Transformer Language Models for Indian Languages](https://mlsanity.com/paper/2011.02323/) (2020)
- [New Alignment Methods for Discriminative Book Summarization](https://mlsanity.com/paper/1305.1319/) (2013)
- [An Improved Text Sentiment Classification Model Using TF-IDF and Next Word Negation](https://mlsanity.com/paper/1806.06407/) (2018)
- [Comparative Study of Long Document Classification](https://mlsanity.com/paper/2111.00702/) (2021)
- [Multi-class Text Classification using BERT-based Active Learning](https://mlsanity.com/paper/2104.14289/) (2021)
- [A Comparative Study on TF-IDF feature Weighting Method and its Analysis using Unstructured Dataset](https://mlsanity.com/paper/2308.04037/) (2023)
- [L3Cube-IndicNews: News-based Short Text and Long Document Classification Datasets in Indic Languages](https://mlsanity.com/paper/2401.02254/) (2024)
- [Hierarchical Neural Network Approaches for Long Document Classification](https://mlsanity.com/paper/2201.06774/) (2022)
- [L3Cube-MahaNews: News-based Short Text and Long Document Classification Datasets in Marathi](https://mlsanity.com/paper/2404.18216/) (2024)

## Built on by (most cited first, 1 of 1)

- [Efficient Zero-Shot Long Document Classification by Reducing Context Through Sentence Ranking](https://mlsanity.com/paper/2508.17490/) (2025)

## Abstract (arXiv)

Long document classification poses challenges due to the computational limitations of transformer-based models, particularly BERT, which are constrained by fixed input lengths and quadratic attention complexity. Moreover, using the full document for classification is often redundant, as only a subset of sentences typically carries the necessary information. To address this, we propose a TF-IDF-based sentence ranking method that improves efficiency by selecting the most informative content. Our approach explores fixed-count and percentage-based sentence selection, along with an enhanced scoring strategy combining normalized TF-IDF scores and sentence length. Evaluated on the MahaNews LDC dataset of long Marathi news articles, the method consistently outperforms baselines such as first, last, and random sentence selection. With MahaBERT-v2, we achieve near-identical classification accuracy with just a 0.33 percent drop compared to the full-context baseline, while reducing input size by over 50 percent and inference latency by 43 percent. This demonstrates that significant context reduction is possible without sacrificing performance, making the method practical for real-world long document classification tasks.