# Shortcut Sequence Tagging

arXiv: https://arxiv.org/abs/1701.00576 · HTML: https://mlsanity.com/paper/1701.00576/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Jan 2017.

## Built on by (most cited first, 1 of 1)

- [Generating CCG Categories](https://mlsanity.com/paper/2103.08139/) (2021)

## Abstract (arXiv)

Deep stacked RNNs are usually hard to train. Adding shortcut connections across different layers is a common way to ease the training of stacked networks. However, extra shortcuts make the recurrent step more complicated. To simply the stacked architecture, we propose a framework called shortcut block, which is a marriage of the gating mechanism and shortcuts, while discarding the self-connected part in LSTM cell. We present extensive empirical experiments showing that this design makes training easy and improves generalization. We propose various shortcut block topologies and compositions to explore its effectiveness. Based on this architecture, we obtain a 6% relatively improvement over the state-of-the-art on CCGbank supertagging dataset. We also get comparable results on POS tagging task.