mlsanityData to 7 Oct 2026

Transformadores: Fundamentos teoricos y Aplicaciones

arXiv 2302.09327 ↗First posted 18 Feb 2023cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Transformers are a neural network architecture originally developed for natural language processing, which have since become a foundational tool for solving a wide range of problems, including text, audio, image processing, reinforcement learning, and other tasks involving heterogeneous input data. Their hallmark is the self-attention mechanism, which allows the model to weigh different parts of the input sequence dynamically, and is an evolution of earlier attention-based approaches. This article provides readers with the necessary background to understand recent research on transformer models, and presents the mathematical and algorithmic foundations of their core components. It also explores the architecture's various elements, potential modifications, and some of the most relevant applications. The article is written in Spanish to help make this scientific knowledge more accessible to the Spanish-speaking community.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 62 of its 82 references that are in the map. First posted on arXiv 18 Feb 2023.

Builds on

62 of 82 references.

References · 62

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  6. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  7. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  8. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  9. 2020End-to-End Object Detection with Transformerscited by 7,384
  10. 2017Mask R-CNNcited by 7,259
  11. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  12. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
38 more
  1. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  3. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  4. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  5. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  6. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  7. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  8. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  9. 2021ViViT: A Video Vision Transformercited by 1,610
  10. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,368
  11. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,148
  12. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,131
  13. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,096
  14. 2020Pre-Trained Image Processing Transformercited by 891
  15. 2022A Generalist Agentcited by 802
  16. 2017Focal Loss for Dense Object Detectioncited by 749
  17. 2019VideoBERT: A Joint Model for Video and Language Representation Learningcited by 702
  18. 2019Multimodal Transformer for Unaligned Multimodal Language Sequencescited by 637
  19. 2018Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidatescited by 632
  20. 2020GraphCodeBERT: Pre-training Code Representations with Data Flowcited by 600
  21. 2022DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETRcited by 569
  22. 2021AI Choreographer: Music Conditioned 3D Dance Generation with AIST++cited by 479
  23. 2021VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Textcited by 427
  24. 2021Conditional DETR for Fast Training Convergencecited by 422
  25. 2022Multimodal Learning with Transformers: A Surveycited by 399
  26. 2020Meta Pseudo Labelscited by 372
  27. 2020Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease predictioncited by 368
  28. 2021Attention Bottlenecks for Multimodal Fusioncited by 361
  29. 2021Pix2seq: A Language Modeling Framework for Object Detectioncited by 345
  30. 2020Visual Transformers: Token-based Image Representation and Processing for Computer Visioncited by 333
  31. 2016FastText.zip: Compressing text classification modelscited by 331
  32. 2022Learning Audio-Visual Speech Representation by Masked Multimodal Cluster Predictioncited by 277
  33. 2016Using the Output Embedding to Improve Language Modelscited by 266
  34. 2020Attention is All You Need in Speech Separationcited by 184
  35. 2021Fast Convergence of DETR with Spatially Modulated Co-Attentioncited by 176
  36. 2022DaViT: Dual Attention Vision Transformerscited by 174
  37. 2015End-to-end people detection in crowded scenescited by 140
  38. 2019Supervised Multimodal Bitransformers for Classifying Images and Textcited by 130

50 most cited of 62 All 62 (JSON)

Built on by

Not cited yet.