mlsanityData to 14 Oct 2026

Concept-Centric Token Interpretation for Vector-Quantized Generative Models

arXiv 2506.00698 ↗First posted 31 May 2025cs.CV cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Vector-Quantized Generative Models (VQGMs) have emerged as powerful tools for image generation. However, the key component of VQGMs -- the codebook of discrete tokens -- is still not well understood, e.g., which tokens are critical to generate an image of a certain concept? This paper introduces Concept-Oriented Token Explanation (CORTEX), a novel approach for interpreting VQGMs by identifying concept-specific token combinations. Our framework employs two methods: (1) a sample-level explanation method that analyzes token importance scores in individual images, and (2) a codebook-level explanation method that explores the entire codebook to find globally relevant tokens. Experimental results demonstrate CORTEX's efficacy in providing clear explanations of token usage in the generative process, outperforming baselines across multiple pretrained VQGMs. Besides enhancing VQGMs transparency, CORTEX is useful in applications such as targeted image editing and shortcut feature detection. Our code is available at https://github.com/YangTianze009/CORTEX.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 35 of its 44 references that are in the map. First posted on arXiv 31 May 2025.

Builds on

35 of 44 references.

References · 35

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  4. 2017Neural Discrete Representation Learningcited by 4,712
  5. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  6. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  7. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  8. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
  9. 2017SmoothGrad: removing noise by adding noisecited by 1,109
  10. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,023
  11. 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 776
  12. 2021Vector-quantized Image Modeling with Improved VQGANcited by 658
23 more
  1. 2020Transformer Interpretability Beyond Attention Visualizationcited by 627
  2. 2019Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networkscited by 582
  3. 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 492
  4. 2023Finite Scalar Quantization: VQ-VAE Made Simplecited by 440
  5. 2022MAGVIT: Masked Generative Video Transformercited by 346
  6. 2015Understanding Neural Networks Through Deep Visualizationcited by 334
  7. 2022MAGE: MAsked Generative Encoder to Unify Representation Learning and Image Synthesiscited by 227
  8. 2023Interpreting CLIP's Image Representation via Text-Based Decompositioncited by 196
  9. 2016Synthesizing the preferred inputs for neurons in neural networks via deep generator networkscited by 181
  10. 2016Multifaceted Feature Visualization: Uncovering the Different Types of Features Learned By Each Neuron in Deep Neural Networkscited by 106
  11. 2024Video-LaVIT: Unified Video-Language Pre-training with Decoupled Visual-Motional Tokenizationcited by 100
  12. 2023Unified Language-Vision Pretraining in LLM with Dynamic Discrete Visual Tokenizationcited by 87
  13. 2024Usable XAI: 10 Strategies Towards Exploiting Explainability in the LLM Eracited by 46
  14. 2019Explaining a black-box using Deep Variational Information Bottleneck Approachcited by 36
  15. 2018Towards Explanation of DNN-based Prediction with Guided Feature Inversioncited by 31
  16. 2021How Well do Feature Visualizations Support Causal Understanding of CNN Activations?cited by 28
  17. 2023Unlocking Feature Visualization for Deeper Networks with MAgnitude Constrained Optimizationcited by 27
  18. 2018End-to-End Saliency Mapping via Probability Distribution Predictioncited by 17
  19. 2022Visual Concepts Tokenizationcited by 14
  20. 2025Enhancing Cognition and Explainability of Multimodal Foundation Models with Self-Synthesized Datacited by 12
  21. 2022Saliency Cards: A Framework to Characterize and Compare Saliency Methodscited by 11
  22. 2022New Definitions and Evaluations for Saliency Methods: Staying Intrinsic, Complete and Soundcited by 10
  23. 2024Probabilistic Conceptual Explainers: Trustworthy Conceptual Explanations for Vision Foundation Modelscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025RAU: Reference-based Anatomical Understanding with Vision Language Modelscited by 3
  2. 2026Online Skill Learning for Web Agents via State-Grounded Dynamic Retrievalcited by 3
  3. 2025RSTR: Reducing SpatioTemporal Redundancy in Diffusion Transformerscited by 1
  4. 2026Self-Improving Small Object Grounding in LVLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Image Understanding Makes for A Good Tokenizer for Image Generation
  2. 2025UniTok: A Unified Tokenizer for Visual Generation and Understanding
  3. 2024TokenFlow: Unified Image Tokenizer for Multimodal Understanding and Generation
  4. 2025Ming-UniVision: Joint Image Understanding and Generation with a Unified Continuous Tokenizer
  5. 2025Bridging Continuous and Discrete Tokens for Autoregressive Visual Generation
  6. 2026Vision Foundation Models as Generalist Tokenizers for Image Generation
  7. 2025A Token-level Text Image Foundation Model for Document Understanding
  8. 2024Explainable Concept Generation through Vision-Language Preference Learning for Understanding Neural Networks' Internal Representations