mlsanityData to 14 Oct 2026

Probing the 3D Object-Level Understanding of Pre-Trained Detection Transformers

arXiv 2608.01495 ↗First posted 2 Aug 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Detection transformer models, including DETR and its extensions, learn to output a set of object-level embeddings that can be simultaneously decoded into 2D bounding boxes and class distributions. In this paper, we investigate what pre-trained 2D detection transformers understand about the 3D properties of objects. Specifically, we investigate the extent to which properties including the depth of objects from the camera and the 3D location of objects relative to the camera can be recovered from object-level embeddings using linear and non-linear probes. Across a range of detection transformer models, our results show a surprisingly strong and previously unknown ability of 2D DETR models to represent useful information about the 3D properties of objects, despite the complete lack of 3D supervision during model pre-training.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 31 of its 37 references that are in the map. First posted on arXiv 2 Aug 2026.

Builds on

31 of 37 references.

References · 31

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  6. 2020End-to-End Object Detection with Transformerscited by 7,414
  7. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  8. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
  9. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  10. 2021Vision Transformers for Dense Predictioncited by 1,803
  11. 2024Depth Anything V2cited by 1,422
  12. 2016Understanding intermediate layers using linear classifier probescited by 1,353
19 more
  1. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,152
  2. 2021Transformers in Vision: A Surveycited by 1,013
  3. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 876
  4. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 716
  5. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
  6. 2023The Internal State of an LLM Knows When It's Lyingcited by 659
  7. 2021DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queriescited by 658
  8. 2023DETRs Beat YOLOs on Real-time Object Detectioncited by 533
  9. 2021Conditional DETR for Fast Training Convergencecited by 424
  10. 2020Virtual KITTI 2cited by 353
  11. 2024Probing the 3D Awareness of Visual Foundation Modelscited by 177
  12. 2024RT-DETRv2: Improved Baseline with Bag-of-Freebies for Real-Time Detection Transformercited by 91
  13. 2022MonoDETR: Depth-guided Transformer for Monocular 3D Object Detectioncited by 86
  14. 2024LW-DETR: A Transformer Replacement to YOLO for Real-Time Detectioncited by 36
  15. 2023A General Protocol to Probe Large Vision Models for 3D Physical Understandingcited by 32
  16. 2024DepthCues: Evaluating Monocular Depth Perception in Large Vision Modelscited by 19
  17. 2023Evaluating the Stability of Semantic Concept Representations in CNNs for Robust Explainabilitycited by 11
  18. 2025How to Probe: Simple Yet Effective Techniques for Improving Post-hoc Explanationscited by 8
  19. 2023Evaluating Representations with Readout Model Switchingcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Revisiting DETR Pre-training for Object Detection
  2. 2024ImageNet3D: Towards General-Purpose Object-Level 3D Understanding
  3. 20233DiffTection: 3D Object Detection with Geometry-Aware Diffusion Features
  4. 2024Knowledge Distillation via Query Selection for Detection Transformer
  5. 2022Open World DETR: Transformer based Open World Object Detection
  6. 2024Believing is Seeing: Unobserved Object Detection using Generative Models
  7. 2025State Space Model Meets Transformer: A New Paradigm for 3D Object Detection
  8. 2025LocateAnything3D: Vision-Language 3D Detection with Chain-of-Sight