mlsanityData to 14 Oct 2026

Fourier Controller Networks for Real-Time Decision-Making in Embodied Learning

arXiv 2405.19885 ↗First posted 30 May 2024cs.LG cs.ROcited by 3

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Transformer has shown promise in reinforcement learning to model time-varying features for obtaining generalized low-level robot policies on diverse robotics datasets in embodied learning. However, it still suffers from the issues of low data efficiency and high inference latency. In this paper, we propose to investigate the task from a new perspective of the frequency domain. We first observe that the energy density in the frequency domain of a robot's trajectory is mainly concentrated in the low-frequency part. Then, we present the Fourier Controller Network (FCNet), a new network that uses Short-Time Fourier Transform (STFT) to extract and encode time-varying features through frequency domain interpolation. In order to do real-time decision-making, we further adopt FFT and Sliding DFT methods in the model architecture to achieve parallel training and efficient recurrent inference. Extensive results in both simulated (e.g., D4RL) and real-world environments (e.g., robot locomotion) demonstrate FCNet's substantial efficiency and effectiveness over existing methods such as Transformer, e.g., FCNet outperforms Transformer on multi-environmental robotics datasets of all types of sizes (from 1.9M to 120M). The project page and code can be found https://thkkk.github.io/fcnet.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 62 of its 74 references that are in the map. First posted on arXiv 30 May 2024.

Builds on

62 of 74 references.

References · 62

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2017Proximal Policy Optimization Algorithmscited by 15,215
  4. 2016Layer Normalizationcited by 5,669
  5. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  6. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  7. 2018Neural Ordinary Differential Equationscited by 3,779
  8. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  9. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  10. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  11. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,908
  12. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
38 more
  1. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  2. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  3. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 2,022
  4. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,850
  5. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  6. 2020Reformer: The Efficient Transformercited by 1,668
  7. 2019Generating Long Sequences with Sparse Transformerscited by 1,546
  8. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  9. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,427
  10. 2020Gradient Surgery for Multi-Task Learningcited by 1,256
  11. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
  12. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 862
  13. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 840
  14. 2023RWKV: Reinventing RNNs for the Transformer Eracited by 827
  15. 2022A Generalist Agentcited by 808
  16. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 782
  17. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 743
  18. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 715
  19. 2018Learning Dexterous In-Hand Manipulationcited by 687
  20. 2019Behavior Regularized Offline Reinforcement Learningcited by 662
  21. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 644
  22. 2023Retentive Network: A Successor to Transformer for Large Language Modelscited by 613
  23. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 604
  24. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 512
  25. 2022Hungry Hungry Hippos: Towards Language Modeling with State Space Modelscited by 499
  26. 2022VIMA: General Robot Manipulation with Multimodal Promptscited by 407
  27. 2023Hyena Hierarchy: Towards Larger Convolutional Language Modelscited by 393
  28. 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperationcited by 391
  29. 2016Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updatescited by 353
  30. 2021FNet: Mixing Tokens with Fourier Transformscited by 347
  31. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 229
  32. 2022Legged Locomotion in Challenging Terrains using Egocentric Visioncited by 151
  33. 2022Walk These Ways: Tuning Robot Control for Generalization with Multiplicity of Behaviorcited by 138
  34. 2023RT-Trajectory: Robotic Task Generalization via Hindsight Trajectory Sketchescited by 133
  35. 2023Q-Transformer: Scalable Offline Reinforcement Learning via Autoregressive Q-Functionscited by 131
  36. 2021Why Generalization in RL is Difficult: Epistemic POMDPs and Implicit Partial Observabilitycited by 109
  37. 2020Context-aware Dynamics Model for Generalization in Model-Based Reinforcement Learningcited by 98
  38. 2022Physics-Informed Machine Learning: A Survey on Problems, Methods and Applicationscited by 92

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 1.

3 papers.

  1. 2024ManiBox: Enhancing Embodied Spatial Generalization via Scalable Simulation Data Generationscited by 4
  2. 2026Hyper-DP3: Frequency-Aware Right-Sizing of 3D Diffusion Policies for Visuomotor Controlcited by 3
  3. 2025Decision SpikeFormer: Spike-Driven Transformer for Decision Makingcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026Learning Nonlinear Systems In-Context: From Synthetic Data to Real-World Motor Control
  2. 2026Fourier Features Let Agents Learn High Precision Policies with Imitation Learning
  3. 2024Advances in Transformers for Robotic Applications: A Review
  4. 2025Wavelet Fourier Diffuser: Frequency-Aware Diffusion Model for Reinforcement Learning
  5. 2026Learning Smooth Time-Varying Linear Policies with an Action Jacobian Penalty
  6. 2026Articulated-Body Dynamics Network: Dynamics-Grounded Prior for Robot Learning
  7. 2020Neural Dynamic Policies for End-to-End Sensorimotor Learning
  8. 2026One for All: A Non-Linear Transformer can Enable Cross-Domain Generalization for In-Context Reinforcement Learning