depyf: Open the Opaque Box of PyTorch Compiler for Machine Learning Researchers
Not on the map. Why?
Abstract (arXiv)
PyTorch texttt2.x introduces a compiler designed to accelerate deep learning programs. However, for machine learning researchers, adapting to the PyTorch compiler to full potential can be challenging. The compiler operates at the Python bytecode level, making it appear as an opaque box. To address this, we introduce textttdepyf, a tool designed to demystify the inner workings of the PyTorch compiler. textttdepyf decompiles bytecode generated by PyTorch back into equivalent source code, and establishes connections between in-memory code objects and their on-disk source code counterparts. This feature enables users to step through the source code line by line using debuggers, thus enhancing their understanding of the underlying processes. Notably, textttdepyf is non-intrusive and user-friendly, primarily relying on two convenient context managers for its core functionality. The project is hrefhttps://github.com/thuml/depyf openly available and is recognized as a hrefhttps://pytorch.org/ecosystem/PyTorch ecosystem project.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 80 of its 100 references that are in the map. First posted on arXiv 14 Mar 2024.
Builds on
80 of 100 references.
References · 80
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
- 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
- 2016Densely Connected Convolutional Networkscited by 6,720
- 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
38 more
- 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
- 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
- 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
- 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
- 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
- 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
- 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,811
- 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
- 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,546
- 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
- 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
- 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
- 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,054
- 2019Searching for MobileNetV3cited by 1,994
- 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
- 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
- 2020YOLOv4: Optimal Speed and Accuracy of Object Detectioncited by 1,630
- 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,446
- 2019Deep High-Resolution Representation Learning for Visual Recognitioncited by 1,420
- 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,311
- 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,200
- 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,102
- 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,060
- 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,035
- 2020Designing Network Design Spacescited by 960
- 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 943
- 2021CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classificationcited by 806
- 2021Going deeper with Image Transformerscited by 764
- 2021Twins: Revisiting the Design of Spatial Attention in Vision Transformerscited by 565
- 2020Beyond English-Centric Multilingual Machine Translationcited by 561
- 2019LayoutLM: Pre-training of Text and Layout for Document Image Understandingcited by 553
- 2021RepVGG: Making VGG-style ConvNets Great Againcited by 522
- 2017Progressive Neural Architecture Searchcited by 487
- 2018FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Searchcited by 485
- 2021ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biasescited by 484
- 2020MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devicescited by 482
- 2021LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inferencecited by 475
- 2019Res2Net: A New Multi-scale Backbone Architecturecited by 470
50 most cited of 80 All 80 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.