mlsanityData to 7 Oct 2026

Lite DETR : An Interleaved Multi-Scale Encoder for Efficient DETR

arXiv 2303.07335 ↗First posted 13 Mar 2023cs.CVcited by 36

Not on the map. Why?

Abstract (arXiv)

Recent DEtection TRansformer-based (DETR) models have obtained remarkable performance. Its success cannot be achieved without the re-introduction of multi-scale feature fusion in the encoder. However, the excessively increased tokens in multi-scale features, especially for about 75% of low-level features, are quite computationally inefficient, which hinders real applications of DETR models. In this paper, we present Lite DETR, a simple yet efficient end-to-end object detection framework that can effectively reduce the GFLOPs of the detection head by 60% while keeping 99% of the original performance. Specifically, we design an efficient encoder block to update high-level features (corresponding to small-resolution feature maps) and low-level features (corresponding to large-resolution feature maps) in an interleaved way. In addition, to better fuse cross-scale features, we develop a key-aware deformable attention to predict more reliable attention weights. Comprehensive experiments validate the effectiveness and efficiency of the proposed Lite DETR, and the efficient encoder strategy can generalize well across existing DETR-based models. The code will be available in urlhttps://github.com/IDEA-Research/Lite-DETR.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). Builds on 32 of its 38 references that are in the map. First posted on arXiv 13 Mar 2023.

Builds on

32 of 38 references.

References · 32

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  5. 2020End-to-End Object Detection with Transformerscited by 7,384
  6. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  7. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  8. 2018YOLOv3: An Incremental Improvementcited by 2,652
  9. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  10. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,306
  11. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,148
  12. 2019EfficientDet: Scalable and Efficient Object Detectioncited by 1,018
20 more
  1. 2022YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectorscited by 978
  2. 2018Path Aggregation Network for Instance Segmentationcited by 796
  3. 2022DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETRcited by 569
  4. 2022DN-DETR: Accelerate DETR Training by Introducing Query DeNoisingcited by 486
  5. 2019Hybrid Task Cascade for Instance Segmentationcited by 479
  6. 2021Conditional DETR for Fast Training Convergencecited by 422
  7. 2022Mask DINO: Towards A Unified Transformer-based Framework for Object Detection and Segmentationcited by 341
  8. 2019NAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detectioncited by 289
  9. 2020DetectoRS: Detecting Objects with Recursive Feature Pyramid and Switchable Atrous Convolutioncited by 222
  10. 2022DETRs with Hybrid Matchingcited by 149
  11. 2021Efficient DETR: Improving End-to-End Object Detector with Dense Priorcited by 128
  12. 2021Sparse DETR: Efficient End-to-End Object Detection with Learnable Sparsitycited by 98
  13. 2022AdaMixer: A Fast-Converging Query-Based Object Detectorcited by 78
  14. 2021PnP-DETR: Towards Efficient Visual Analysis with Transformerscited by 52
  15. 2021End-to-End Trainable Multi-Instance Pose Estimation with Transformerscited by 30
  16. 2022D^2ETR: Decoder-Only DETR with Computationally Efficient Cross-Scale Attentioncited by 12
  17. 2022Towards Efficient Use of Multi-Scale Features in Transformer-Based Object Detectorscited by 10
  18. 2022An Extendable, Efficient and Effective Transformer-based Object Detectorcited by 5
  19. 2022Efficient Decoder-free Object Detection with Transformerscited by 5
  20. 2022DETR++: Taming Your Multi-Scale Detection Transformercited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 9; 2024: 17; 2025: 5; 2026: 5.

36 papers.

Sort:

By area:

  1. 2023DETRs Beat YOLOs on Real-time Object Detectioncited by 526
  2. 2023Transformer-Based Visual Segmentation: A Surveycited by 124
  3. 2024Grounding DINO 1.5: Advance the "Edge" of Open-Set Object Detectioncited by 122
  4. 2024DEIM: DETR with Improved Matching for Fast Convergencecited by 67
  5. 2024TAPTR: Tracking Any Point with Transformers as Detectioncited by 43
  6. 2023Vote2Cap-DETR++: Decoupling Localization and Describing for End-to-End 3D Dense Captioningcited by 38
  7. 2024LW-DETR: A Transformer Replacement to YOLO for Real-Time Detectioncited by 36
  8. 2024Salience DETR: Enhancing Detection Transformer with Hierarchical Salience Filtering Refinementcited by 30
  9. 2023BAM-DETR: Boundary-Aligned Moment Detection Transformer for Temporal Sentence Grounding in Videoscited by 29
  10. 2024TAPTRv2: Attention-based Position Update Improves Tracking Any Pointcited by 18
  11. 2023Hausdorff Distance Matching with Adaptive Query Denoising for Rotated Detection Transformercited by 9
  12. 2024TAPTRv3: Spatial and Temporal Context Foster Robust Tracking of Any Point in Long Videocited by 8
24 more
  1. 2023Align-DETR: Enhancing End-to-end Object Detection with Aligned Losscited by 5
  2. 2024LeanGaussian: Breaking Pixel or Point Cloud Correspondence in Modeling 3D Gaussianscited by 3
  3. 2024CASPFormer: Trajectory Prediction from BEV Images with Deformable Attentioncited by 3
  4. 2025Self-Supervised Visual Prompting for Cross-Domain Road Damage Detectioncited by 3
  5. 2023Box-DETR: Understanding and Boxing Conditional Spatial Queriescited by 2
  6. 2023DECO: Unleashing the Potential of ConvNets for Query-based Detection and Segmentationcited by 2
  7. 2024DI-MaskDINO: A Joint Object Detection and Instance Segmentation Modelcited by 2
  8. 2024Referencing Where to Focus: Improving VisualGrounding with Referential Querycited by 2
  9. 2026Don't let the information slip awaycited by 2
  10. 2024Synchronized Object Detection for Autonomous Sorting, Mapping, and Quantification of Materials in Circular Healthcarecited by 1
  11. 2024Global Clipper: Enhancing Safety and Reliability of Transformer-based Object Detection Modelscited by 1
  12. 2024Mr. DETR++: Instructive Multi-Route Training for Detection Transformers with Mixture-of-Expertscited by 1
  13. 2025MOBIUS: Big-to-Mobile Universal Instance Segmentation via Multi-modal Bottleneck Fusion and Calibrated Decoder Pruningcited by 1
  14. 2026CP-WSP: A Declarative CP-SAT Framework for Configurable Multi-Constraint Workforce Schedulingcited by 1
  15. 2023Efficient Pre-training for Localized Instruction Generation of Videoscited by 0
  16. 2024Progressive Token Length Scaling in Transformer Encoders for Efficient Universal Segmentationcited by 0
  17. 2024Image-Specific Adaptation of Transformer Encoders for Compute-Efficient Segmentationcited by 0
  18. 2024Cross Resolution Encoding-Decoding For Detection Transformerscited by 0
  19. 2025A lightweight model FDM-YOLO for small target improvement based on YOLOv8cited by 0
  20. 2025CMEdataset Advancing China Map Detection and Standardization with Digital Image Resourcescited by 0
  21. 2025Decoupled PROB: Decoupled Query Initialization Tasks and Objectness-Class Learning for Open World Object Detectioncited by 0
  22. 2026Prior-Guided DETR for Ultrasound Nodule Detectioncited by 0
  23. 2026Corrupting Attention: Evasion-Based Adversarial Attacks on Encoder Attention in Detection Transformerscited by 0
  24. 2026Enhanced Knowledge Distillation for Detection Transformer via Teacher Prediction Refinementcited by 0