mlsanityData to 7 Oct 2026

Attention when you need

arXiv 2501.07440 ↗First posted 13 Jan 2025q-bio.NC cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Paying attention improves performance, but attention is metabolically costly, so how should a resource-efficient agent allocate it? We study optimal allocation strategies using a normative model of a signal detection task in which attention comes at a cost. The model reveals that optimal attention is temporally structured in one of two patterns depending on task conditions: when attention costs penalize intense focus, optimal attention ramps up as evidence for the signal accumulates, but with less prohibitive attention costs, optimal attention fluctuates rhythmically. In cases with rhythmic attention, allocation frequency increases with task parameters such as reward magnitude for successful detection, signal brevity, and signal frequency. We argue that rhythmic attention emerges naturally in an agent whose belief updates are dominated by a stable temporal prior and not by the likelihood of new sensory observations. Our results characterize the conditions under which rhythmic vs. ramping attention is optimal and offer a normative account of attentional fluctuations observed in sustained attention tasks.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 16 of its 56 references that are in the map. First posted on arXiv 13 Jan 2025.

Builds on

16 of 56 references.

References · 16

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  3. 2018SlowFast Networks for Video Recognitioncited by 1,862
  4. 2016Temporal Segment Networks: Towards Good Practices for Deep Action Recognitioncited by 899
  5. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 791
  6. 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 591
  7. 2016Adaptive Computation Time for Recurrent Neural Networkscited by 479
  8. 2014Recurrent Models of Visual Attentioncited by 344
  9. 2024Mixture-of-Depths: Dynamically allocating compute in transformer-based language modelscited by 242
  10. 2021PonderNet: Learning to Pondercited by 156
  11. 2018AdaFrame: Adaptive Frame Selection for Fast Video Recognitioncited by 134
  12. 2019SCSampler: Sampling Salient Clips from Video for Efficient Action Recognitioncited by 133
4 more
  1. 2017Skip RNN: Learning to Skip State Updates in Recurrent Neural Networkscited by 49
  2. 2020Active Measure Reinforcement Learning for Observation Cost Minimizationcited by 17
  3. 2023Act-Then-Measure: Reinforcement Learning for Partially Observable Environments with Active Measuringcited by 10
  4. 2024Conditional computation in neural networks: principles and research trendscited by 10

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.