# Image Compression for Machine and Human Vision with Spatial-Frequency Adaptation

arXiv: https://arxiv.org/abs/2407.09853 · HTML: https://mlsanity.com/paper/2407.09853/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 63 of its 76 references that are in the map.
- First posted on arXiv 13 Jul 2024.

## Builds on (most cited first, 30 of 63)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [Visual Prompt Tuning](https://mlsanity.com/paper/2203.12119/) (2022)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018)
- [Towards a Unified View of Parameter-Efficient Transfer Learning](https://mlsanity.com/paper/2110.04366/) (2021)
- [Segmenter: Transformer for Semantic Segmentation](https://mlsanity.com/paper/2105.05633/) (2021)
- [AdapterFusion: Non-Destructive Task Composition for Transfer Learning](https://mlsanity.com/paper/2005.00247/) (2020)
- [AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognition](https://mlsanity.com/paper/2205.13535/) (2022)
- [End-to-end Optimized Image Compression](https://mlsanity.com/paper/1611.01704/) (2016)
- [MaPLe: Multi-modal Prompt Learning](https://mlsanity.com/paper/2210.03117/) (2022)
- [Joint Autoregressive and Hierarchical Priors for Learned Image Compression](https://mlsanity.com/paper/1809.02736/) (2018)
- [Learned Image Compression with Discretized Gaussian Mixture Likelihoods and Attention Modules](https://mlsanity.com/paper/2001.01568/) (2020)
- [Vision Transformer Adapter for Dense Predictions](https://mlsanity.com/paper/2205.08534/) (2022)
- [Deep Learning for Person Re-identification: A Survey and Outlook](https://mlsanity.com/paper/2001.04193/) (2020)

## Built on by (most cited first, 26 of 26)

- [Rate-Distortion-Cognition Controllable Versatile Neural Image Compression](https://mlsanity.com/paper/2407.11700/) (2024)
- [Feature Complementation Architecture for Visual Place Recognition](https://mlsanity.com/paper/2506.12401/) (2025)
- [Diff-ICMH: Harmonizing Machine and Human Vision in Image Compression with Generative Prior](https://mlsanity.com/paper/2511.22549/) (2025)
- [Explicit Residual-Based Scalable Image Coding for Humans and Machines](https://mlsanity.com/paper/2506.19297/) (2025)
- [When MLLMs Meet Compression Distortion: A Coding Paradigm Tailored to MLLMs](https://mlsanity.com/paper/2509.24258/) (2025)
- [SLIM: Semantic-based Low-bitrate Image compression for Machines by leveraging diffusion](https://mlsanity.com/paper/2512.18200/) (2025)
- [Guided Diffusion for the Extension of Machine Vision to Human Visual Perception](https://mlsanity.com/paper/2503.17907/) (2025)
- [Compression Tells Intelligence: Visual Coding, Visual Token Technology, and the Unification](https://mlsanity.com/paper/2601.20742/) (2026)
- [All-in-One Transferring Image Compression from Human Perception to Multi-Machine Perception](https://mlsanity.com/paper/2504.12997/) (2025)
- [Perception-Oriented Latent Coding for High-Performance Compressed Domain Semantic Inference](https://mlsanity.com/paper/2507.01608/) (2025)
- [Machines Serve Human: A Novel Variable Human-machine Collaborative Compression Framework](https://mlsanity.com/paper/2511.08915/) (2025)
- [Progressive Learned Image Compression for Machine Perception](https://mlsanity.com/paper/2512.20070/) (2025)
- [Efficient Masked Image Compression with Position-Indexed Self-Attention](https://mlsanity.com/paper/2504.12923/) (2025)
- [Comp-X: On Defining an Interactive Learned Image Compression Paradigm With Expert-driven LLM Agent](https://mlsanity.com/paper/2508.15243/) (2025)
- [ISCS: Parameter-Guided Feature Pruning for Resource-Constrained Embodied Perception](https://mlsanity.com/paper/2509.16853/) (2025)
- [Codebook-Based Adaptive Feature Compression With Semantic Enhancement for Edge-Cloud Systems](https://mlsanity.com/paper/2509.18481/) (2025)
- [Generative Semantic Coding for Ultra-Low Bitrate Visual Communication and Analysis](https://mlsanity.com/paper/2510.27324/) (2025)
- [Error-Propagation-Free Learned Video Compression With Dual-Domain Progressive Temporal Alignment](https://mlsanity.com/paper/2512.10450/) (2025)
- [Benchmarking and Enhancing VLM for Compressed Image Understanding](https://mlsanity.com/paper/2512.20901/) (2025)
- [FAAR: Efficient Frequency-Aware Multi-Task Fine-Tuning via Automatic Rank Selection](https://mlsanity.com/paper/2603.20403/) (2026)
- [CI-ICM: Channel Importance-driven Learned Image Coding for Machines](https://mlsanity.com/paper/2604.05347/) (2026)
- [Multi-task Just Recognizable Difference for Video Coding for Machines: Database, Model, and Coding Application](https://mlsanity.com/paper/2604.09421/) (2026)
- [What and Where to Adapt: Structure-Semantics Co-Tuning for Machine Vision Compression via Synergistic Adapters](https://mlsanity.com/paper/2604.10017/) (2026)
- [Coarse-to-Fine: Progressive Image Compression for Semantically Hierarchical Classification](https://mlsanity.com/paper/2605.08266/) (2026)
- [MoECodec: Image Compression for joint human and machine perception via Mixture-of-Experts](https://mlsanity.com/paper/2606.21033/) (2026)
- [CrossMambaTuning: Synergistic Spatial and Cross-Layer Adaptation for Machine Vision Compression](https://mlsanity.com/paper/2608.25568/) (2026)

## Abstract (arXiv)

Image compression for machine and human vision (ICMH) has gained increasing attention in recent years. Existing ICMH methods are limited by high training and storage overheads due to heavy design of task-specific networks. To address this issue, in this paper, we develop a novel lightweight adapter-based tuning framework for ICMH, named Adapt-ICMH, that better balances task performance and bitrates with reduced overheads. We propose a spatial-frequency modulation adapter (SFMA) that simultaneously eliminates non-semantic redundancy with a spatial modulation adapter, and enhances task-relevant frequency components and suppresses task-irrelevant frequency components with a frequency modulation adapter. The proposed adapter is plug-and-play and compatible with almost all existing learned image compression models without compromising the performance of pre-trained models. Experiments demonstrate that Adapt-ICMH consistently outperforms existing ICMH frameworks on various machine vision tasks with fewer fine-tuned parameters and reduced computational complexity. Code will be released at https://github.com/qingshi9974/ECCV2024-AdpatICMH .