# A Structured Literature Review on Traditional Approaches in Current Natural Language Processing

arXiv: https://arxiv.org/abs/2505.12970 · HTML: https://mlsanity.com/paper/2505.12970/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 94 references that are in the map.
- First posted on arXiv 19 May 2025.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examples](https://mlsanity.com/paper/2310.01469/) (2023)
- [Few-NERD: A Few-Shot Named Entity Recognition Dataset](https://mlsanity.com/paper/2105.07464/) (2021)
- [Evaluating ChatGPT's Information Extraction Capabilities: An Assessment of Performance, Explainability, Calibration, and Faithfulness](https://mlsanity.com/paper/2304.11633/) (2023)
- [DEGREE: A Data-Efficient Generation-Based Event Extraction Model](https://mlsanity.com/paper/2108.12724/) (2021)
- [A Survey on Text Classification: From Shallow to Deep Learning](https://mlsanity.com/paper/2008.00364/) (2020)
- [Multi-document Summarization via Deep Learning Techniques: A Survey](https://mlsanity.com/paper/2011.04843/) (2020)
- [Joint Entity and Relation Extraction with Set Prediction Networks](https://mlsanity.com/paper/2011.01675/) (2020)
- [Zero-shot Temporal Relation Extraction with ChatGPT](https://mlsanity.com/paper/2304.05454/) (2023)
- [Analyzing the Performance of Large Language Models on Code Summarization](https://mlsanity.com/paper/2404.08018/) (2024)
- [Language Models for German Text Simplification: Overcoming Parallel Data Scarcity through Style-specific Pre-training](https://mlsanity.com/paper/2305.12908/) (2023)
- [Improving Faithfulness of Large Language Models in Summarization via Sliding Generation and Self-Consistency](https://mlsanity.com/paper/2407.21443/) (2024)
- [Reviewing Labels: Label Graph Network with Top-k Prediction Set for Relation Extraction](https://mlsanity.com/paper/2212.14270/) (2022)
- [Teaching the Pre-trained Model to Generate Simple Texts for Text Simplification](https://mlsanity.com/paper/2305.12463/) (2023)
- [Fusing Multimodal Signals on Hyper-complex Space for Extreme Abstractive Text Summarization (TL;DR) of Scientific Contents](https://mlsanity.com/paper/2306.13968/) (2023)
- [WOT-Class: Weakly Supervised Open-world Text Classification](https://mlsanity.com/paper/2305.12401/) (2023)
- [AgentRE: An Agent-Based Framework for Navigating Complex Information Landscapes in Relation Extraction](https://mlsanity.com/paper/2409.01854/) (2024)
- [Automating the Information Extraction from Semi-Structured Interview Transcripts](https://mlsanity.com/paper/2403.04819/) (2024)
- [A Comparative Study of Hybrid Models in Health Misinformation Text Classification](https://mlsanity.com/paper/2410.06311/) (2024)
- [A General Contextualized Rewriting Framework for Text Summarization](https://mlsanity.com/paper/2207.05948/) (2022)
- [SimpLex: a lexical text simplification architecture](https://mlsanity.com/paper/2304.07002/) (2023)
- [Quantitative Information Extraction from Humanitarian Documents](https://mlsanity.com/paper/2408.04941/) (2024)

## Abstract (arXiv)

The continued rise of neural networks and large language models in the more recent past has altered the natural language processing landscape, enabling new approaches towards typical language tasks and achieving mainstream success. Despite the huge success of large language models, many disadvantages still remain and through this work we assess the state of the art in five application scenarios with a particular focus on the future perspectives and sensible application scenarios of traditional and older approaches and techniques. In this paper we survey recent publications in the application scenarios classification, information and relation extraction, text simplification as well as text summarization. After defining our terminology, i.e., which features are characteristic for traditional techniques in our interpretation for the five scenarios, we survey if such traditional approaches are still being used, and if so, in what way they are used. It turns out that all five application scenarios still exhibit traditional models in one way or another, as part of a processing pipeline, as a comparison/baseline to the core model of the respective paper, or as the main model(s) of the paper. For the complete statistics, see https://zenodo.org/records/13683801