# Optimal-er Auctions through Attention

arXiv: https://arxiv.org/abs/2202.13110 · HTML: https://mlsanity.com/paper/2202.13110/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 81 references that are in the map.
- First posted on arXiv 26 Feb 2022.

## Builds on (most cited first, 30 of 42)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention](https://mlsanity.com/paper/2006.16236/) (2020)
- [Reformer: The Efficient Transformer](https://mlsanity.com/paper/2001.04451/) (2020)
- [Linformer: Self-Attention with Linear Complexity](https://mlsanity.com/paper/2006.04768/) (2020)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet](https://mlsanity.com/paper/2101.11986/) (2021)
- [Efficient Attention: Attention with Linear Complexities](https://mlsanity.com/paper/1812.01243/) (2018)
- [Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flow](https://mlsanity.com/paper/1810.00821/) (2018)
- [Deep Models of Interactions Across Sets](https://mlsanity.com/paper/1803.02879/) (2018)
- [Real-Time Bidding with Multi-Agent Reinforcement Learning in Display Advertising](https://mlsanity.com/paper/1802.09756/) (2018)
- [The Sample Complexity of Revenue Maximization](https://mlsanity.com/paper/1502.00963/) (2015)
- [Complexity of Mechanism Design](https://mlsanity.com/paper/1408.1486/) (2014)
- [Automated Mechanism Design via Neural Networks](https://mlsanity.com/paper/1805.03382/) (2018)
- [A Permutation-Equivariant Neural Network Architecture For Auction Design](https://mlsanity.com/paper/2003.01497/) (2020)
- [Neural Auction: End-to-End Learning of Auction Mechanisms for E-Commerce Advertising](https://mlsanity.com/paper/2106.03593/) (2021)
- [Auction learning as a two-player game](https://mlsanity.com/paper/2006.05684/) (2020)
- [Sample Complexity of Automated Mechanism Design](https://mlsanity.com/paper/1606.04145/) (2016)
- [Settling the Sample Complexity of Single-parameter Revenue Maximization](https://mlsanity.com/paper/1904.04962/) (2019)
- [Learning Simple Auctions](https://mlsanity.com/paper/1604.03171/) (2016)
- [PreferenceNet: Encoding Human Preferences in Auction Design with Deep Learning](https://mlsanity.com/paper/2106.03215/) (2021)
- [Differentiable Economics for Randomized Affine Maximizer Auctions](https://mlsanity.com/paper/2202.02872/) (2022)
- [ProportionNet: Balancing Fairness and Revenue for Auction Design with Deep Learning](https://mlsanity.com/paper/2010.06398/) (2020)
- [A Context-Integrated Transformer-Based Neural Network for Auction Design](https://mlsanity.com/paper/2201.12489/) (2022)
- [Strong Duality for a Multiple-Good Monopolist](https://mlsanity.com/paper/1409.4150/) (2014)
- [The Sample Complexity of Up-to-varepsilon Multi-Dimensional Revenue Maximization](https://mlsanity.com/paper/1808.02458/) (2018)

## Built on by (most cited first, 27 of 27)

- [A Scalable Neural Network for DSIC Affine Maximizer Auction Design](https://mlsanity.com/paper/2305.12162/) (2023)
- [GemNet: Menu-Based, Strategy-Proof Multi-Bidder Auctions Through Deep Learning](https://mlsanity.com/paper/2406.07428/) (2024)
- [Information Design in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2305.06807/) (2023)
- [Principal-Agent Reinforcement Learning: Orchestrating AI Agents with Contracts](https://mlsanity.com/paper/2407.18074/) (2024)
- [Benefits of Permutation-Equivariance in Auction Mechanisms](https://mlsanity.com/paper/2210.05579/) (2022)
- [Data Market Design through Deep Learning](https://mlsanity.com/paper/2310.20096/) (2023)
- [Mediated Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2306.08419/) (2023)
- [Learning Payment-Free Resource Allocation Mechanisms](https://mlsanity.com/paper/2311.10927/) (2023)
- [Are Equivariant Equilibrium Approximators Beneficial?](https://mlsanity.com/paper/2301.11481/) (2023)
- [Automated Deterministic Auction Design with Objective Decomposition](https://mlsanity.com/paper/2402.11904/) (2024)
- [Social Environment Design](https://mlsanity.com/paper/2402.14090/) (2024)
- [Deep Incentive Design with Differentiable Equilibrium Blocks](https://mlsanity.com/paper/2603.07705/) (2026)
- [Neural Auctions Compromise Bidder Information](https://mlsanity.com/paper/2303.00116/) (2023)
- [Deep Reinforcement Learning for Sequential Combinatorial Auctions](https://mlsanity.com/paper/2407.08022/) (2024)
- [Bridging the Gap Between Estimated and True Regret Towards Reliable Regret Estimation in Deep Learning based Mechanism Design](https://mlsanity.com/paper/2601.13489/) (2026)
- [Deep Learning Meets Mechanism Design: Key Results and Some Novel Applications](https://mlsanity.com/paper/2401.05683/) (2024)
- [Statistically Truthful Auctions via Acceptance Rule](https://mlsanity.com/paper/2405.12016/) (2024)
- [Achieving PAC Guarantees in Mechanism Design through Multi-Armed Bandits](https://mlsanity.com/paper/2412.00345/) (2024)
- [Regularized Proportional Fairness Mechanism for Resource Allocation Without Money](https://mlsanity.com/paper/2501.01111/) (2025)
- [Optimal Auction Design in the Joint Advertising](https://mlsanity.com/paper/2507.07418/) (2025)
- [Universal Representation of Generalized Convex Functions and their Gradients](https://mlsanity.com/paper/2509.04477/) (2025)
- [Strategic Cyber Defense via Reinforcement Learning-Guided Combinatorial Auctions](https://mlsanity.com/paper/2509.10983/) (2025)
- [Open Problems in Differentiable Social Choice: Learning Mechanisms, Decisions, and Alignment](https://mlsanity.com/paper/2602.03003/) (2026)
- [LLM Active Alignment: A Nash Equilibrium Perspective](https://mlsanity.com/paper/2602.06836/) (2026)
- [Enhancing Affine Maximizer Auctions with Correlation-Aware Payment](https://mlsanity.com/paper/2602.09455/) (2026)
- [MenuNet: A Strategy-Proof Mechanism for Matching Markets](https://mlsanity.com/paper/2605.03216/) (2026)
- [Transformer-Enhanced Reinforcement Learning: Fundamentals and Applications in Communication Networks](https://mlsanity.com/paper/2606.05208/) (2026)

## Abstract (arXiv)

RegretNet is a recent breakthrough in the automated design of revenue-maximizing auctions. It combines the flexibility of deep learning with the regret-based approach to relax the Incentive Compatibility (IC) constraint (that participants prefer to bid truthfully) in order to approximate optimal auctions. We propose two independent improvements of RegretNet. The first is a neural architecture denoted as RegretFormer that is based on attention layers. The second is a loss function that requires explicit specification of an acceptable IC violation denoted as regret budget. We investigate both modifications in an extensive experimental study that includes settings with constant and inconstant number of items and participants, as well as novel validation procedures tailored to regret-based approaches. We find that RegretFormer consistently outperforms RegretNet in revenue (i.e. is optimal-er) and that our loss function both simplifies hyperparameter tuning and allows to unambiguously control the revenue-regret trade-off by selecting the regret budget.