mlsanityData to 14 Oct 2026

Rationally Inattentive Inverse Reinforcement Learning Explains YouTube Commenting Behavior

arXiv 1910.11703 ↗First posted 24 Oct 2019cs.LG cs.SI stat.MLcited by 6

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

We consider a novel application of inverse reinforcement learning with behavioral economics constraints to model, learn and predict the commenting behavior of YouTube viewers. Each group of users is modeled as a rationally inattentive Bayesian agent which solves a contextual bandit problem. Our methodology integrates three key components. First, to identify distinct commenting patterns, we use deep embedded clustering to estimate framing information (essential extrinsic features) that clusters users into distinct groups.Second, we present an inverse reinforcement learning algorithm that uses Bayesian revealed preferences to test for rationality: does there exist a utility function that rationalizes the given data, and if yes, can it be used to predict commenting behavior? Finally, we impose behavioral economics constraints stemming from rational inattention to characterize the attention span of groups of users. The test imposes a R'enyi mutual information cost constraint which impacts how the agent can select attention strategies to maximize their expected utility. After a careful analysis of a massive YouTube dataset, our surprising result is that in most YouTube user groups, the commenting behavior is consistent with optimizing a Bayesian utility with rationally inattentive constraints. The paper also highlights how the rational inattention model can accurately predict commenting behavior. The massive YouTube dataset and analysis used in this paper are available on GitHub and completely reproducible.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 10 of its 79 references that are in the map. First posted on arXiv 24 Oct 2019.

Builds on

10 of 79 references.

References · 10

  1. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
  2. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
  3. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  4. 2016Deep Networks with Stochastic Depthcited by 979
  5. 2015Unsupervised Deep Embedding for Clustering Analysiscited by 767
  6. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 490
  7. 2017Inverse Reward Designcited by 249
  8. 2015A Primer on Neural Network Models for Natural Language Processingcited by 76
  9. 2014Resourceful Contextual Banditscited by 45
  10. 2014MatConvNet - Convolutional Neural Networks for MATLABcited by 36

Built on by

Cited by year

Citing papers per year: 2020: 1; 2021: 2; 2022: 2; 2023: 0; 2024: 0; 2025: 1.

6 papers.

  1. 2025Behavioral Homophily in Social Media via Inverse Reinforcement Learning: A Reddit Case Studycited by 6
  2. 2021Learning the Optimal Recommendation from Explorative Userscited by 3
  3. 2020Necessary and Sufficient Conditions for Inverse Reinforcement Learning of Bayesian Stopping Time Problemscited by 2
  4. 2022Solving Dynamic Principal-Agent Problems with a Rationally Inattentive Principalcited by 2
  5. 2021Rationally Inattentive Utility Maximization for Interpretable Deep Image Classificationcited by 0
  6. 2022Machine Learning enabled models for YouTube Ranking Mechanism and Views Predictioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2018Estimating Rationally Inattentive Utility Functions with Deep Clustering for Framing - Applications in YouTube Engagement Dynamics ↗
  2. 2023Adversarial Batch Inverse Reinforcement Learning: Learn to Reward from Imperfect Demonstration for Interactive Recommendation
  3. 2025Inverse Reinforcement Learning using Revealed Preferences and Passive Stochastic Optimization
  4. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress
  5. 2024Quantifying the Sensitivity of Inverse Reinforcement Learning to Misspecification
  6. 2024Bayesian Inverse Reinforcement Learning for Non-Markovian Rewards
  7. 2026A Lecture Note on Offline RL and IRL, Part II: Foundations of Inverse Reinforcement Learning and Dynamic Discrete Choice Models
  8. 2024The Virtues of Pessimism in Inverse Reinforcement Learning