mlsanityData to 14 Oct 2026

Selective Amortization of Full-Budget Counterfactual Reasoning for Visual Token Communication

arXiv 2609.30756 ↗First posted 25 Sep 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Generative image communication transmits compact semantic tokens under a limited packet budget, where token selection directly affects the final reconstruction quality after the complete packet is decoded. However, accurately estimating the terminal value of every candidate token requires repeated receiver-side reconstruction, resulting in substantial encoder-side computation. To address this problem, we propose ACV-Gate, an adaptive candidate evaluation framework that learns to approximate full-budget counterfactual evaluation and selectively assigns exact evaluations to the most informative candidates. Specifically, a set-aware student is trained using terminal advantages and regrets to predict candidate rankings directly, while a selective refinement mechanism evaluates only a bounded candidate set containing both Local-MDL and direct actions; cost-based thresholds further enable explicit control of the average evaluation workload. Experiments on CIFAR-10 show that ACV-Gate consistently improves reconstruction quality while substantially reducing candidate evaluations; at 0.20 bpp, the primary adaptive configuration improves PSNR over LocalMDL by 0.636 dB with only 2.13 candidate evaluations per image, corresponding to 27.60% of the calls required by the Exact-Full expert. Matched-candidate comparisons, synchronized GPU measurements, and evaluations on STL-10 and 384 *384 scale transfer further demonstrate consistent quality computation trade-offs, with particularly pronounced gains at low bit rates. These results show that combining terminal-value learning with selective candidate evaluation provides an effective and controllable mechanism for allocating encoder computation in packet-constrained generative image communication.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 26 of its 32 references that are in the map. First posted on arXiv 25 Sep 2026.

Builds on

26 of 32 references.

References · 26

  1. 2017Neural Discrete Representation Learningcited by 4,712
  2. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  3. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  4. 2022MaskGIT: Masked Generative Image Transformercited by 1,099
  5. 2022Token Merging: Your ViT But Fastercited by 1,021
  6. 2018Variational image compression with a scale hyperpriorcited by 935
  7. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 864
  8. 2017Selective Classification for Deep Neural Networkscited by 852
  9. 2016End-to-end Optimized Image Compressioncited by 696
  10. 2018Joint Autoregressive and Hierarchical Priors for Learned Image Compressioncited by 664
  11. 2024LLaVA-PruMerge: Adaptive Token Reduction for Efficient Large Multimodal Modelscited by 402
  12. 2024SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inferencecited by 391
14 more
  1. 2022Not All Patches are What You Need: Expediting Vision Transformers via Token Reorganizationscited by 363
  2. 2021AdaViT: Adaptive Tokens for Efficient Vision Transformercited by 362
  3. 2020Deep Learning Enabled Semantic Communication Systemscited by 292
  4. 2020Consistent Estimators for Learning to Defer to an Expertcited by 271
  5. 2025DivPrune: Diversity-based Visual Token Pruning for Large Multimodal Modelscited by 181
  6. 2024Beyond Text-Visual Attention: Exploiting Visual Cues for Effective Token Pruning in VLMscited by 157
  7. 2024Fit and Prune: Fast and Training-free Visual Token Pruning for Multi-modal Large Language Modelscited by 126
  8. 2025What Kind of Visual Tokens Do We Need? Training-free Visual Token Pruning for Multi-modal Large Language Models from the Perspective of Graphcited by 36
  9. 2023Learning to Defer with Limited Expert Predictionscited by 25
  10. 2025SCOPE: Saliency-Coverage Oriented Token Pruning for Efficient Multimodel LLMscited by 24
  11. 2025When Token Pruning is Worse than Random: Understanding Visual Token Information in VLLMscited by 12
  12. 2022Generative Model Based Highly Efficient Semantic Communication Approach for Image Transmissioncited by 11
  13. 2025SQ-GAN: Semantic Image Communications Using Masked Vector Quantizationcited by 4
  14. 2026Baseline-Relative Counterfactual Refinement for Bit-Aware Visual Token Communicationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026RUTA: Principled Visual Token Allocation via Rate-Utility Optimization
  2. 2026The Model Knows Which Tokens Matter: Automatic Token Selection via Noise Gating
  3. 2026Adaptive Visual Token Reduction for Accelerated Image Understanding
  4. 2026MergeTok: Unified Continuous and Discrete Visual Tokenization via Token Merging
  5. 2025HiViS: Hiding Visual Tokens from the Drafter for Speculative Decoding in Vision-Language Models
  6. 2024HiRED: Attention-Guided Token Dropping for Efficient Inference of High-Resolution Vision-Language Models
  7. 2023Text-Conditioned Sampling Framework for Text-to-Image Generation with Masked Generative Models
  8. 2025Token Sequence Compression for Efficient Multimodal Computing