Thesis of Tomas Kocak

Sequential learning with similarities

This thesis studies several extensions of multi-armed bandit problem, where a learner sequentially selects an action and obtain the reward of the action. Traditionally, the only information the learner acquire is about the obtained reward while information about other actions is hidden from the learner. This limited feedback can be restrictive in some applications like recommender systems, internet advertising, packet routing, etc. Usually, these problems come with structure, similarities between users or actions, additional observations, or any additional assumptions. Therefore, it is natural to incorporate these assumptions to the algorithms to improve their performance. This thesis focuses on multi-armed bandit problem with some underlying structure usually represented by a graph with actions as vertices. First, we study a problem where the graph captures similarities between actions; connected actions tend to grand similar rewards. Second, we study a problem where the learner observes rewards of all the neighbors of the selected action. We study these problems under several additional assumptions on rewards (stochastic, adversarial), side observations (adversarial, stochastic, noisy), actions (one node at the time, several nodes forming a combinatorial structure in the graph). The main contribution of this thesis is to design algorithms for previously mentioned problems together with theoretical and empirical guaranties. We also introduce several novel quantities, to capture the difficulty of some problems, like effective dimension and effective independence number.


Directeur de thèse : Michal VALKO Rapporteurs : András GYÖRGY, Claudio GENTILE Examinateurs : Olivier CAPPÉ, Rémi MUNOS

Thesis of the team defended on 28/11/2016