# Controlling Out-of-Domain Gaps in LLMs for Genre Classification and Generated Text Detection

arXiv: https://arxiv.org/abs/2412.20595 · HTML: https://mlsanity.com/paper/2412.20595/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 38 references that are in the map.
- First posted on arXiv 29 Dec 2024.

## Builds on (most cited first, 15 of 15)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [GLTR: Statistical Detection and Visualization of Generated Text](https://mlsanity.com/paper/1906.04043/) (2019)
- [Label Words are Anchors: An Information Flow Perspective for Understanding In-Context Learning](https://mlsanity.com/paper/2305.14160/) (2023)
- [Topic Modeling in Embedding Spaces](https://mlsanity.com/paper/1907.04907/) (2019)
- [The Science of Detecting LLM-Generated Texts](https://mlsanity.com/paper/2303.07205/) (2023)
- [Revisiting Out-of-distribution Robustness in NLP: Benchmark, Analysis, and LLMs Evaluations](https://mlsanity.com/paper/2306.04618/) (2023)
- [Language Models for Text Classification: Is In-Context Learning Enough?](https://mlsanity.com/paper/2403.17661/) (2024)
- [Adapt in Contexts: Retrieval-Augmented Domain Adaptation via In-Context Learning](https://mlsanity.com/paper/2311.11551/) (2023)
- [LLM-Detector: Improving AI-Generated Chinese Text Detection with Open-Source LLM Instruction Tuning](https://mlsanity.com/paper/2402.01158/) (2024)
- [Genre as Weak Supervision for Cross-lingual Dependency Parsing](https://mlsanity.com/paper/2109.04733/) (2021)
- [The GINCO Training Dataset for Web Genre Identification of Documents Out in the Wild](https://mlsanity.com/paper/2201.03857/) (2022)
- [Estimating Confidence of Predictions of Individual Classifiers and Their Ensembles for the Genre Classification Task](https://mlsanity.com/paper/2206.07427/) (2022)
- [BERT Goes Off-Topic: Investigating the Domain Transfer Challenge using Genre Classification](https://mlsanity.com/paper/2311.16083/) (2023)

## Built on by (most cited first, 3 of 3)

- [Do Large Language Models Truly Grasp Addition? A Rule-Focused Diagnostic Using Two-Integer Arithmetic](https://mlsanity.com/paper/2504.05262/) (2025)
- [SENTRA: Selected-Next-Token Transformer for LLM Text Detection](https://mlsanity.com/paper/2509.12385/) (2025)
- [What Transfers Under Source Shift? Definitions, Examples, and Fine-Tuning for Climate Disclosure Classification](https://mlsanity.com/paper/2607.17952/) (2026)

## Abstract (arXiv)

This study demonstrates that the modern generation of Large Language Models (LLMs, such as GPT-4) suffers from the same out-of-domain (OOD) performance gap observed in prior research on pre-trained Language Models (PLMs, such as BERT). We demonstrate this across two non-topical classification tasks: 1) genre classification and 2) generated text detection. Our results show that when demonstration examples for In-Context Learning (ICL) come from one domain (e.g., travel) and the system is tested on another domain (e.g., history), classification performance declines significantly. To address this, we introduce a method that controls which predictive indicators are used and which are excluded during classification. For the two tasks studied here, this ensures that topical features are omitted, while the model is guided to focus on stylistic rather than content-based attributes. This approach reduces the OOD gap by up to 20 percentage points in a few-shot setup. Straightforward Chain-of-Thought (CoT) methods, used as the baseline, prove insufficient, while our approach consistently enhances domain transfer performance.