mlsanityData to 14 Oct 2026

QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension

arXiv 1804.09541 ↗First posted 23 Apr 2018cs.CL cs.AI cs.LGcited by 164

Not on the map. Why?

Abstract (arXiv)

Current end-to-end machine reading and question answering (Q&A) models are primarily based on recurrent neural networks (RNNs) with attention. Despite their success, these models are often slow for both training and inference due to the sequential nature of RNNs. We propose a new Q&A architecture called QANet, which does not require recurrent networks: Its encoder consists exclusively of convolution and self-attention, where convolution models local interactions and self-attention models global interactions. On the SQuAD dataset, our model is 3x to 13x faster in training and 4x to 9x faster in inference, while achieving equivalent accuracy to recurrent models. The speed-up gain allows us to train the model with much more data. We hence combine our model with data generated by backtranslation from a neural machine translation model. On the SQuAD dataset, our single model, trained with augmented data, achieves 84.6 F1 score on the test set, which is significantly better than the best published F1 score of 81.8.

Key facts. Cited by 164 papers in the map (data through 14 Oct 2026). First posted on arXiv 23 Apr 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 65; 2022: 53; 2023: 23; 2024: 17; 2025: 5; 2026: 1.

164 papers.

Sort:

By area:

  1. 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 538
  2. 2024RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrievalcited by 532
  3. 2021Conditional Positional Encodings for Vision Transformerscited by 406
  4. 2021Mesh Graphormercited by 265
  5. 2021Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answeringcited by 178
  6. 2021Primer: Searching for Efficient Transformers for Language Modelingcited by 164
  7. 2021Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Bettercited by 154
  8. 2021A Survey on Data Augmentation for Text Classificationcited by 111
  9. 2021An Empirical Survey of Data Augmentation for Limited Data Learning in NLPcited by 101
  10. 2022E-Branchformer: Branchformer with Enhanced merging for speech recognitioncited by 92
  11. 2021Boundary Proposal Network for Two-Stage Natural Language Video Localizationcited by 88
  12. 2021Data Augmentation Approaches in Natural Language Processing: A Surveycited by 82
38 more
  1. 2021Towards Zero-Label Language Learningcited by 76
  2. 2022A General Survey on Attention Mechanisms in Deep Learningcited by 64
  3. 2022Capturing Humans in Motion: Temporal-Attentive 3D Human Pose and Shape Estimation from Monocular Videocited by 60
  4. 2021Attention, please! A survey of Neural Attention Models in Deep Learningcited by 51
  5. 2021Knowing More About Questions Can Help: Improving Calibration in Question Answeringcited by 47
  6. 2021Generating Syntactically Controlled Paraphrases without Using Annotated Parallel Pairscited by 45
  7. 2021Generate, Annotate, and Learn: NLP with Synthetic Textcited by 42
  8. 2021Bidirectional Machine Reading Comprehension for Aspect Sentiment Triplet Extractioncited by 40
  9. 2022On Transforming Reinforcement Learning by Transformer: The Development Trajectorycited by 36
  10. 2021FlipDA: Effective and Robust Data Augmentation for Few-Shot Learningcited by 32
  11. 2022Quality Controlled Paraphrase Generationcited by 31
  12. 2022Attention Mechanism in Neural Networks: Where it Comes and Where it Goescited by 29
  13. 2021Natural Language Video Localization with Learnable Moment Proposalscited by 27
  14. 2023Pushing the Limits of ChatGPT on NLP Taskscited by 27
  15. 2021Introspective Distillation for Robust Question Answeringcited by 25
  16. 2022ALLSH: Active Learning Guided by Local Sensitivity and Hardnesscited by 23
  17. 2023ViGT: Proposal-free Video Grounding with Learnable Token in Transformercited by 22
  18. 2022CLMLF:A Contrastive Learning and Multi-Layer Fusion Method for Multimodal Sentiment Detectioncited by 21
  19. 2021Do Question Answering Modeling Improvements Hold Across Benchmarks?cited by 20
  20. 2023Using GPT-4 to Augment Unbalanced Data for Automatic Scoringcited by 20
  21. 2021Network Representation Learning: From Preprocessing, Feature Extraction to Node Embeddingcited by 19
  22. 2021Rethinking Search: Making Domain Experts out of Dilettantescited by 18
  23. 2021Adaptive Hierarchical Graph Reasoning with Semantic Coherence for Video-and-Language Inferencecited by 18
  24. 2021Is Everything in Order? A Simple Way to Order Sentencescited by 15
  25. 2022TreeMix: Compositional Constituency-based Data Augmentation for Natural Language Understandingcited by 15
  26. 2022TASA: Deceiving Question Answering Models by Twin Answer Sentences Attackcited by 15
  27. 2024Regurgitative Training: The Value of Real Data in Training Large Language Modelscited by 15
  28. 2021Towards Socially Intelligent Agents with Mental State Transition and Human Utilitycited by 14
  29. 2021Complex Factoid Question Answering with a Free-Text Knowledge Graphcited by 14
  30. 2021NOAHQA: Numerical Reasoning with Interpretable Graph Question Answering Datasetcited by 14
  31. 2022Simulating Bandit Learning from User Feedback for Extractive Question Answeringcited by 14
  32. 2021Dependency Parsing as MRC-based Span-Span Predictioncited by 13
  33. 2021CaEGCN: Cross-Attention Fusion based Enhanced Graph Convolutional Network for Clusteringcited by 12
  34. 2022GENIUS: Sketch-based Language Model Pre-training via Extreme and Selective Masking for Text Generation and Augmentationcited by 11
  35. 2022Improving Continuous Sign Language Recognition with Consistency Constraints and Signer Removalcited by 11
  36. 2023ParaAMR: A Large-Scale Syntactically Diverse Paraphrase Dataset by AMR Back-Translationcited by 11
  37. 2021Unsupervised Contextual Paraphrase Generation using Lexical Control and Reinforcement Learningcited by 10
  38. 2021Not Far Away, Not So Close: Sample Efficient Nearest Neighbour Data Augmentation via MiniMaxcited by 10

50 most cited of 164 All 164 (JSON)