# Look Ahead Text Understanding and LLM Stitching

arXiv: https://arxiv.org/abs/2412.17836 · HTML: https://mlsanity.com/paper/2412.17836/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 43 references that are in the map.
- First posted on arXiv 16 Dec 2024.

## Builds on (most cited first, 5 of 5)

- [PubMed 200k RCT: a Dataset for Sequential Sentence Classification in Medical Abstracts](https://mlsanity.com/paper/1710.06071/) (2017)
- [Content Planning for Neural Story Generation with Aristotelian Rescoring](https://mlsanity.com/paper/2009.09870/) (2020)
- [Jump to Conclusions: Short-Cutting Transformers With Linear Transformations](https://mlsanity.com/paper/2303.09435/) (2023)
- [Pretrained Language Models for Sequential Sentence Classification](https://mlsanity.com/paper/1909.04054/) (2019)
- [Enhancing Identification of Structure Function of Academic Articles Using Contextual Information](https://mlsanity.com/paper/2111.14110/) (2021)

## Built on by (most cited first, 4 of 4)

- [Large Language Model Empowered Recommendation Meets All-domain Continual Pre-Training](https://mlsanity.com/paper/2504.08949/) (2025)
- [A Theoretical Framework for Modular Learning of Robust Generative Models](https://mlsanity.com/paper/2602.17554/) (2026)
- [Ventriloquist LLMs: Linear Alignment of Late-Stage Representations](https://mlsanity.com/paper/2603.18908/) (2026)
- [Replicating Human Social Perception in Generative AI: Evaluating the Valence-Dominance Model](https://mlsanity.com/paper/2503.04842/) (2025)

## Abstract (arXiv)

This paper proposes a look ahead text understanding problem with look ahead section identification (LASI) as an example. This problem may appear in generative AI as well as human interactions, where we want to understand the direction of a developing text or conversation. We tackle the problem using transformer-based LLMs. We show that LASI is more challenging than classic section identification (SI). We argue that both bidirectional contextual information (e.g., BERT) and unidirectional predictive ability (e.g., GPT) will benefit the task. We propose two approaches to stitch together BERT and GPT. Experiments show that our approach outperforms the established models, especially when there is noise in the text (which is often the case for developing text in generative AI). Our paper sheds light on other look ahead text understanding tasks that are important to social media, such as look ahead sentiment classification, and points out the opportunities to leverage pre-trained LLMs through stitching.