mlsanityData to 14 Oct 2026

Beam Tree Recursive Cells

arXiv 2305.19999 ↗First posted 31 May 2023cs.LG cs.AI cs.CLcited by 6

Not on the map. Why?

Abstract (arXiv)

We propose Beam Tree Recursive Cell (BT-Cell) - a backpropagation-friendly framework to extend Recursive Neural Networks (RvNNs) with beam search for latent structure induction. We further extend this framework by proposing a relaxation of the hard top-k operators in beam search for better propagation of gradient signals. We evaluate our proposed models in different out-of-distribution splits in both synthetic and realistic data. Our experiments show that BTCell achieves near-perfect performance on several challenging structure-sensitive synthetic tasks like ListOps and logical inference while maintaining comparable performance in realistic data against other RvNN-based models. Additionally, we identify a previously unknown failure case for neural models in generalization to unseen number of arguments in ListOps. The code is available at: https://github.com/JRC1995/BeamTreeRecursiveCells.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 54 of its 89 references that are in the map. First posted on arXiv 31 May 2023.

Builds on

54 of 89 references.

References · 54

  1. 2017Attention Is All You Needcited by 60,361
  2. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,739
  3. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
  4. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,079
  5. 2015A large annotated corpus for learning natural language inferencecited by 1,847
  6. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,148
  7. 2020Long Range Arena: A Benchmark for Efficient Transformerscited by 700
  8. 2019Learning the Difference that Makes a Difference with Counterfactually-Augmented Datacited by 439
  9. 2015Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networkscited by 306
  10. 2022Neural Networks and the Chomsky Hierarchycited by 217
  11. 2018Stress Test Evaluation for Natural Language Inferencecited by 213
  12. 2019Stochastic Beams and Where to Find Them: The Gumbel-Top-k Trick for Sampling Sequences Without Replacementcited by 211
38 more
  1. 2020Fast Differentiable Sorting and Rankingcited by 197
  2. 2018ListOps: A Diagnostic Dataset for Latent Tree Learningcited by 129
  3. 2019Differentiable Ranks and Sorting using Optimal Transportcited by 113
  4. 2019Stochastic Optimization of Sorting Networks via Continuous Relaxationscited by 111
  5. 2018Ordered Neurons: Integrating Tree Structures into Recurrent Neural Networkscited by 101
  6. 2018Neural Nearest Neighbors Networkscited by 98
  7. 2020Span-based Semantic Parsing for Compositional Generalizationcited by 77
  8. 2019Tree Transformer: Integrating Tree Structures into Self-Attentioncited by 68
  9. 2021The Neural Data Router: Adaptive Control Flow in Transformers Improves Systematic Generalizationcited by 62
  10. 2019Unsupervised Latent Tree Induction with Deep Inside-Outside Recursive Autoencoderscited by 60
  11. 2015Learning to Transduce with Unbounded Memorycited by 54
  12. 2020Compositional Generalization by Learning Analytical Expressionscited by 53
  13. 2017Learning to Compose Task-Specific Tree Structurescited by 48
  14. 2018The Importance of Being Recurrent for Modeling Hierarchical Structurecited by 42
  15. 2016A Fast Unified Model for Parsing and Sentence Understandingcited by 34
  16. 2020StructFormer: Joint Unsupervised Induction of Dependency and Constituency Structure from Masked Language Modelingcited by 33
  17. 2021Differentiable Sorting Networks for Scalable Sorting and Ranking Supervisioncited by 31
  18. 2021Learning Algebraic Recombination for Compositional Generalizationcited by 31
  19. 2016Learning to Compose Words into Sentences with Reinforcement Learningcited by 30
  20. 2020Tree-structured Attention with Hierarchical Accumulationcited by 30
  21. 2019Cooperative Learning of Disjoint Syntax and Semanticscited by 27
  22. 2022Monotonic Differentiable Sorting Networkscited by 25
  23. 2017Jointly Learning Sentence Embeddings and Syntax with Unsupervised Tree-LSTMscited by 24
  24. 2020Retrofitting Structure-aware Transformer Language Model for End Taskscited by 24
  25. 2015Tree-structured composition in neural networks without tree-structured architecturescited by 23
  26. 2019Ordered Memorycited by 20
  27. 2021R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modelingcited by 19
  28. 2018On Tree-Based Neural Sentence Modelingcited by 16
  29. 2020Latent Compositional Representations Improve Systematic Generalization in Grounded Question Answeringcited by 16
  30. 2021Learning Hierarchical Structures with Differentiable Nondeterministic Stackscited by 16
  31. 2020Learning Context-Free Languages with Nondeterministic Stack RNNscited by 15
  32. 2021Modeling Hierarchical Structures with Continuous Recursive Neural Networkscited by 15
  33. 2022Fast-R2D2: A Pretrained Recursive Neural Network based on Pruned CKY for Grammar Induction and Text Representationcited by 15
  34. 2019A Fully Differentiable Beam Search Decodercited by 14
  35. 2018Backpropagating through Structured Argmax using a SPIGOTcited by 11
  36. 2022Grammar-Based Grounded Lexicon Learningcited by 11
  37. 2015Compositional Distributional Semantics with Long Short Term Memorycited by 10
  38. 2017A Continuous Relaxation of Beam Search for End-to-end Training of Neural Sequence Modelscited by 10

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 3; 2024: 3.

6 papers.

  1. 2023Augmenting Transformers with Recursively Composed Multi-grained Representationscited by 5
  2. 2023Recursion in Recursion: Two-Level Nested Recursion for Length Generalization with Scalabilitycited by 5
  3. 2024Investigating Recurrent Transformers with Dynamic Haltcited by 5
  4. 2023Efficient Beam Tree Recursioncited by 4
  5. 2024Banyan: Improved Representation Learning with Explicit Structurecited by 3
  6. 2024On the Design Space Between Transformers and Recursive Neural Netscited by 1