mlsanityData to 7 Oct 2026

A Decomposable Attention Model for Natural Language Inference

arXiv 1606.01933 ↗First posted 6 Jun 2016cs.CLcited by 214

Not on the map. Why?

Abstract (arXiv)

We propose a simple neural architecture for natural language inference. Our approach uses attention to decompose the problem into subproblems that can be solved separately, thus making it trivially parallelizable. On the Stanford Natural Language Inference (SNLI) dataset, we obtain state-of-the-art results with almost an order of magnitude fewer parameters than previous work and without relying on any word-order information. Adding intra-sentence attention that takes a minimum amount of order into account yields further improvements.

Key facts. Cited by 214 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Jun 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2017: 1; 2018: 1; 2019: 0; 2020: 0; 2021: 80; 2022: 54; 2023: 30; 2024: 31; 2025: 12; 2026: 5.

214 papers.

Sort:

By area:

  1. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  2. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,085
  3. 2021Linear Transformers Are Secretly Fast Weight Programmerscited by 471
  4. 2021Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depthcited by 411
  5. 2021SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarizationcited by 402
  6. 2021Scaling Local Self-Attention for Parameter Efficient Visual Backbonescited by 220
  7. 2021GraphiT: Encoding Graph Structure in Transformerscited by 146
  8. 2021Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Surveycited by 130
  9. 2021A Survey on Data Augmentation for Text Classificationcited by 111
  10. 2021Geometry-Free View Synthesis: Transformers and no 3D Priorscited by 99
  11. 2022M3ViT: Mixture-of-Experts Vision Transformer for Efficient Multi-task Learning with Model-Accelerator Co-designcited by 97
  12. 2023A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?cited by 84
38 more
  1. 2023Hierarchical Video-Moment Retrieval and Step-Captioningcited by 83
  2. 2021DocNLI: A Large-scale Dataset for Document-level Natural Language Inferencecited by 75
  3. 2021Is Attention Better Than Matrix Decomposition?cited by 75
  4. 2021XLM-E: Cross-lingual Language Model Pre-training via ELECTRAcited by 72
  5. 2023EE-LLM: Large-Scale Training and Inference of Early-Exit Large Language Models with 3D Parallelismcited by 69
  6. 2021Towards Robustness Against Natural Language Word Substitutionscited by 68
  7. 2023Transformers as Support Vector Machinescited by 67
  8. 2021Beyond Fully-Connected Layers with Quaternions: Parameterization of Hypercomplex Multiplications with 1/n Parameterscited by 66
  9. 2022Video Transformers: A Surveycited by 63
  10. 2022CoCoNet: Coupled Contrastive Learning Network with Multi-level Feature Ensemble for Multi-modality Image Fusioncited by 63
  11. 2021Fastformer: Additive Attention Can Be All You Needcited by 61
  12. 2021Poolingformer: Long Document Modeling with Pooling Attentioncited by 60
  13. 2023Max-Margin Token Selection in Attention Mechanismcited by 57
  14. 2022Swin Transformer for Fast MRIcited by 56
  15. 2024A Comprehensive Survey of Foundation Models in Medicinecited by 56
  16. 2021Attention, please! A survey of Neural Attention Models in Deep Learningcited by 50
  17. 2024A Survey on Visual Mambacited by 50
  18. 2022MulT: An End-to-End Multitask Learning Transformercited by 47
  19. 2022HeatViT: Hardware-Efficient Adaptive Token Pruning for Vision Transformerscited by 46
  20. 2022Synthetic Disinformation Attacks on Automated Fact Verification Systemscited by 44
  21. 2021Transformer-based Dual Relation Graph for Multi-label Image Recognitioncited by 41
  22. 2021UPDeT: Universal Multi-agent Reinforcement Learning via Policy Decoupling with Transformerscited by 39
  23. 2023What can a Single Attention Layer Learn? A Study Through the Random Features Lenscited by 39
  24. 2022Visual Attention Methods in Deep Learning: An In-Depth Surveycited by 37
  25. 2023On the Optimization and Generalization of Multi-head Attentioncited by 37
  26. 2021RAIN: Reinforced Hybrid Attention Inference Network for Motion Forecastingcited by 36
  27. 2022On Transforming Reinforcement Learning by Transformer: The Development Trajectorycited by 35
  28. 2022Word Order Does Matter (And Shuffled Language Models Know It)cited by 32
  29. 2024The What, Why, and How of Context Length Extension Techniques in Large Language Models -- A Detailed Surveycited by 32
  30. 2021Distributed NLI: Learning to Predict Human Opinion Distributions for Language Reasoningcited by 31
  31. 2021High-Resolution Complex Scene Synthesis with Transformerscited by 31
  32. 2022The Tail Wagging the Dog: Dataset Construction Biases of Social Bias Benchmarkscited by 31
  33. 2023Self-Checker: Plug-and-Play Modules for Fact-Checking with Large Language Modelscited by 31
  34. 2023Mitigating Over-smoothing in Transformers via Regularized Nonlocal Functionalscited by 31
  35. 2021Improving Transformers with Probabilistic Attention Keyscited by 28
  36. 2023A Review of Deep Learning for Video Captioningcited by 28
  37. 2021StARformer: Transformer with State-Action-Reward Representations for Visual Reinforcement Learningcited by 27
  38. 2023Pushing the Limits of ChatGPT on NLP Taskscited by 27

50 most cited of 214 All 214 (JSON)