Equivariant Transduction through Invariant Alignment
Not on the map. Why?
Abstract (arXiv)
The ability to generalize compositionally is key to understanding the potentially infinite number of sentences that can be constructed in a human language from only a finite number of words. Investigating whether NLP models possess this ability has been a topic of interest: SCAN (Lake and Baroni, 2018) is one task specifically proposed to test for this property. Previous work has achieved impressive empirical results using a group-equivariant neural network that naturally encodes a useful inductive bias for SCAN (Gordon et al., 2020). Inspired by this, we introduce a novel group-equivariant architecture that incorporates a group-invariant hard alignment mechanism. We find that our network's structure allows it to develop stronger equivariance properties than existing group-equivariant approaches. We additionally find that it outperforms previous group-equivariant networks empirically on the SCAN task. Our results suggest that integrating group-equivariance into a variety of neural architectures is a potentially fruitful avenue of research, and demonstrate the value of careful analysis of the theoretical properties of such architectures.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 14 of its 20 references that are in the map. First posted on arXiv 22 Sep 2022.
Builds on
14 of 20 references.
References · 14
- 2014Adam: A Method for Stochastic Optimizationcited by 37,240
- 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
- 2016Group Equivariant Convolutional Networkscited by 1,165
- 2015Effective Approaches to Attention-based Neural Machine Translationcited by 1,026
- 2018On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groupscited by 300
- 2019Compositionality decomposed: how do neural networks generalise?cited by 265
- 2017Equivariance Through Parameter-Sharingcited by 141
- 2020A Benchmark for Systematic Generalization in Grounded Language Understandingcited by 108
- 2019Compositional generalization through meta sequence-to-sequence learningcited by 99
- 2019CLOSURE: Assessing Systematic Generalization of CLEVR Modelscited by 63
- 2019Compositional generalization in a deep seq2seq model by separating syntax and semanticscited by 56
- 2020Compositional Generalization by Learning Analytical Expressionscited by 53
2 more
- 2021Lexicon Learning for Few-Shot Neural Sequence Modelingcited by 28
- 2018Hard Non-Monotonic Attention for Character-Level Transductioncited by 12
Built on by
Citing papers per year: 2023: 1; 2024: 2.
3 papers.