# Incremental Processing in the Age of Non-Incremental Encoders: An Empirical Assessment of Bidirectional Models for Incremental NLU

arXiv: https://arxiv.org/abs/2010.05330 · HTML: https://mlsanity.com/paper/2010.05330/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Oct 2020.

## Built on by (most cited first, 14 of 14)

- [Towards Incremental Transformers: An Empirical Analysis of Transformer Models for Incremental NLU](https://mlsanity.com/paper/2109.07364/) (2021)
- [Teaching BERT to Wait: Balancing Accuracy and Latency for Streaming Disfluency Detection](https://mlsanity.com/paper/2205.00620/) (2022)
- [TAPIR: Learning Adaptive Revision for Incremental Natural Language Understanding with a Two-Pass Model](https://mlsanity.com/paper/2305.10845/) (2023)
- [Efficient Encoders for Streaming Sequence Tagging](https://mlsanity.com/paper/2301.09244/) (2023)
- [Real-time Caller Intent Detection In Human-Human Customer Support Spoken Conversations](https://mlsanity.com/paper/2208.06802/) (2022)
- [Sentence-Incremental Neural Coreference Resolution](https://mlsanity.com/paper/2305.16947/) (2023)
- [The Road to Quality is Paved with Good Revisions: A Detailed Evaluation Methodology for Revision Policies in Incremental Sequence Labelling](https://mlsanity.com/paper/2307.15508/) (2023)
- [On the Challenges of Fully Incremental Neural Dependency Parsing](https://mlsanity.com/paper/2309.16254/) (2023)
- [Prior Lessons of Incremental Dialogue and Robot Action Management for the Age of Language Models](https://mlsanity.com/paper/2501.00953/) (2025)
- [Learning to generate and corr- uh I mean repair language in real-time](https://mlsanity.com/paper/2308.11683/) (2023)
- [From Partial to Strictly Incremental Constituent Parsing](https://mlsanity.com/paper/2402.02782/) (2024)
- [When Only Time Will Tell: Interpreting How Transformers Process Local Ambiguities Through the Lens of Restart-Incrementality](https://mlsanity.com/paper/2402.13113/) (2024)
- [Revising with a Backward Glance: Regressions and Skips during Reading as Cognitive Signals for Revision Policies in Incremental Processing](https://mlsanity.com/paper/2310.18229/) (2023)
- [GesBench: When and Where Do Multimodal Language Models Perceive Social Meaning?](https://mlsanity.com/paper/2610.15129/) (2026)

## Abstract (arXiv)

While humans process language incrementally, the best language encoders currently used in NLP do not. Both bidirectional LSTMs and Transformers assume that the sequence that is to be encoded is available in full, to be processed either forwards and backwards (BiLSTMs) or as a whole (Transformers). We investigate how they behave under incremental interfaces, when partial output must be provided based on partial input seen up to a certain time step, which may happen in interactive systems. We test five models on various NLU datasets and compare their performance using three incremental evaluation metrics. The results support the possibility of using bidirectional encoders in incremental mode while retaining most of their non-incremental quality. The "omni-directional" BERT model, which achieves better non-incremental performance, is impacted more by the incremental access. This can be alleviated by adapting the training regime (truncated training), or the testing procedure, by delaying the output until some right context is available or by incorporating hypothetical right contexts generated by a language model like GPT-2.