Recommendation System-based Upper Confidence Bound for Online Advertising

Nhan Nguyen-Thanh, Dana Marinca, Kinda Khawam, David Rohde, Flavian Vasile, Elena Simona Lohan, Steven Martin, Dominique Quadri

Research output: Chapter in Book/Report/Conference proceedingConference contributionProfessional


In this paper, the method UCB-RS, which resorts to recommendation system (RS) for enhancing the upper-confidence bound algorithm UCB, is presented. The proposed method is used for dealing with non-stationary and large-state spaces multi-armed bandit problems. The proposed method has been targeted to the problem of the product recommendation in the online advertising. Through extensive testing with RecoGym, an OpenAI Gym-based reinforcement learning environment for the product recommendation in online advertising, the proposed method outperforms the widespread reinforcement learning schemes such as $\epsilon$-Greedy, Upper Confidence (UCB1) and Exponential Weights for Exploration and Exploitation (EXP3).
Original languageEnglish
Title of host publicationREVEAL 2019
Publication statusPublished - 9 Sept 2019
Publication typeD3 Professional conference proceedings


  • cs.IR
  • cs.LG
  • stat.ML


Dive into the research topics of 'Recommendation System-based Upper Confidence Bound for Online Advertising'. Together they form a unique fingerprint.

Cite this