mlsanityData to 14 Oct 2026

The Road to Quality is Paved with Good Revisions: A Detailed Evaluation Methodology for Revision Policies in Incremental Sequence Labelling

arXiv 2307.15508 ↗First posted 28 Jul 2023cs.CLcited by 2

Not on the map. Why?

Abstract (arXiv)

Incremental dialogue model components produce a sequence of output prefixes based on incoming input. Mistakes can occur due to local ambiguities or to wrong hypotheses, making the ability to revise past outputs a desirable property that can be governed by a policy. In this work, we formalise and characterise edits and revisions in incremental sequence labelling and propose metrics to evaluate revision policies. We then apply our methodology to profile the incremental behaviour of three Transformer-based encoders in various tasks, paving the road for better revision policies.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 16 of its 54 references that are in the map. First posted on arXiv 28 Jul 2023.

Builds on

16 of 54 references.

References · 16

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfacescited by 377
  3. 2020SimulEval: An Evaluation Toolkit for Simultaneous Translationcited by 94
  4. 2020Simultaneous Translation Policies: From Fixed to Adaptivecited by 43
  5. 2020Re-translation versus Streaming for Simultaneous Translationcited by 31
  6. 2021Towards Incremental Transformers: An Empirical Analysis of Transformer Models for Incremental NLUcited by 17
  7. 2020Incremental Processing in the Age of Non-Incremental Encoders: An Empirical Assessment of Bidirectional Models for Incremental NLUcited by 14
  8. 2019Annotation and Classification of Sentence-level Revision Improvementcited by 12
  9. 2022Teaching BERT to Wait: Balancing Accuracy and Latency for Streaming Disfluency Detectioncited by 12
  10. 2018Incremental Natural Language Processing: Challenges, Strategies, and Evaluationcited by 8
  11. 2020Opportunistic Decoding with Timely Correction for Simultaneous Translationcited by 8
  12. 2023TAPIR: Learning Adaptive Revision for Incremental Natural Language Understanding with a Two-Pass Modelcited by 5
4 more
  1. 2019Assessing incrementality in sequence-to-sequence modelscited by 4
  2. 2023Efficient Encoders for Streaming Sequence Taggingcited by 4
  3. 2022Real-time Caller Intent Detection In Human-Human Customer Support Spoken Conversationscited by 2
  4. 2023Sentence-Incremental Neural Coreference Resolutioncited by 2

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 1.

2 papers.

  1. 2024When Only Time Will Tell: Interpreting How Transformers Process Local Ambiguities Through the Lens of Restart-Incrementalitycited by 1
  2. 2023Revising with a Backward Glance: Regressions and Skips during Reading as Cognitive Signals for Revision Policies in Incremental Processingcited by 0