# Instruction-Conditioned Electromagnetic Spectrum Understanding via Budget-Adaptive Signal Tokenization

arXiv: https://arxiv.org/abs/2610.12142 · HTML: https://mlsanity.com/paper/2610.12142/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 48 references that are in the map.
- First posted on arXiv 8 Oct 2026.

## Placed in

- Multimodal LLMs / vision-language models

## Builds on (most cited first, 14 of 14)

- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Qwen3-VL Technical Report](https://mlsanity.com/paper/2511.21631/) (2025)
- [VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understanding](https://mlsanity.com/paper/2501.13106/) (2025)
- [Kimi-Audio Technical Report](https://mlsanity.com/paper/2504.18425/) (2025)
- [Ultralytics YOLO Evolution: An Overview of YOLO27, YOLO26, YOLO11, YOLOv8, and YOLOv5 Object Detectors for Computer Vision and Pattern Recognition](https://mlsanity.com/paper/2510.09653/) (2025)
- [Large Scale Radio Frequency Signal Classification](https://mlsanity.com/paper/2207.09918/) (2022)
- [RadioLLM: Introducing Large Language Model into Cognitive Radio via Hybrid Prompt and Token Reprogrammings](https://mlsanity.com/paper/2501.17888/) (2025)
- [RF-GPT: Teaching AI to See the Wireless World](https://mlsanity.com/paper/2602.14833/) (2026)
- [Seeing Radio: From Zero RF Priors to Explainable Modulation Recognition with Vision Language Models](https://mlsanity.com/paper/2601.13157/) (2026)
- [Large Scale Radio Frequency Wideband Signal Detection & Recognition](https://mlsanity.com/paper/2211.10335/) (2022)
- [PReD: An LLM-based Foundation Multimodal Model for Electromagnetic Perception, Recognition, and Decision](https://mlsanity.com/paper/2603.28183/) (2026)
- [RIS-MAE: A Self-Supervised Modulation Classification Method Based on Raw IQ Signals and Masked Autoencoder](https://mlsanity.com/paper/2508.00274/) (2025)
- [RF-Analyzer: Can Vision-Language Models Learn RF Understanding from Synthetic Data?](https://mlsanity.com/paper/2605.04676/) (2026)

## Abstract (arXiv)

Electromagnetic spectrum monitoring increasingly requires flexible analysis beyond task-specific recognition and detection. Multimodal large language models offer a unified interface, but extending vision-language models (VLMs) to raw I/Q signals requires tokenization that balances fidelity against a strict budget. For signals, dense encoding causes token costs to grow with observation length, whereas fixed-resolution compression may discard short-duration or localized signal evidence. Thus, we propose BATok, a budget-adaptive signal tokenizer that adjusts token capacity to the input length while allocating that capacity according to the signal content. BATok constructs candidate representations from signal-derived features using lightweight multi-resolution branches, then combines a local energy prior with learnable queries to resample these representations into compact signal tokens. The number of tokens adapts to the input length while remaining strictly bounded. The resulting tokens are projected into the language embedding space of VLMs. We further introduce EMSpec-Instruct, a multimodal instruction dataset aligning raw I/Q signals, waterfall images, and language supervision for modulation recognition, structured detection, and language-conditioned signal grounding. Experiments show that BATok learns effective signal representations and achieves competitive performance across all tasks.