Thesis of Anthony Kobanda

Reinforcement Learning with Structured Representations and Adaptive Control

This thesis investigates how structured representations can make reinforcement learning agents more reliable and more adaptable when data are imperfect, tasks evolve, and resources are limited. It focuses primarily on learning from data and goal-conditioned navigation, where agents must plan over long distances from prerecorded trajectories that may cover only part of the possible situations. A first line of work concerns the geometric structure of the state space. In many environments, reaching one state from another does not have the same cost in both directions. The thesis therefore relies on quasimetrics, that is, directed distances, to represent reachability and the time required to reach a goal. These signals are then transformed into explicit planning structures. Projective Quasimetric Planning and Adaptive Quasimetric Mapping construct sparse sets of keypoints and directed graphs that can be used to plan routes, detect transitions that have become unreliable, and recompute a route when the topology of the environment changes. A second line of work concerns continual learning from data. When an agent learns several tasks sequentially, updating a single policy can lead to forgetting previously acquired skills, while storing a complete policy for each task quickly increases memory requirements. The thesis studies policy subspaces that make it possible to reuse existing behaviors and add new capacity only when necessary. Hierarchical Subspaces of Policies notably separate high-level decisions, related to subgoal selection, from low-level decisions, related to movement execution. Continual NavBench provides task sequences, datasets, and metrics to jointly evaluate performance, forgetting, transfer, memory, and computational costs. A third line of work specifically addresses resource constraints. The Budgeted Decision Transformer learns, from offline data, both which information to acquire and which action to execute under an explicit cost constraint. This makes it possible to study the trade-off between performance and perception cost when some observations are more expensive than others. Finally, the thesis studies an experimental prototype for drone navigation in Rainbow Six Siege. The Quasimetric NavGraph transforms directed time-to-reach estimates into a sparse and inspectable navigation graph built from player traces. This structure makes it possible to localize certain sources of failure and modify routing without retraining the entire controller. Taken together, these works support a common idea: explicitly organizing learned knowledge, in the form of directed distances, graphs, policy families, or information-acquisition rules, facilitates its reuse, adaptation, and analysis in evolving environments under operational constraints.

Jury

- M. Odalric-Ambrym MAILLARD Chargé de recherche Inria Scool, Centre Inria de l'Université de Lille Directeur de thèse, - M. David FILLIAT Directeur de recherche Agence Ministérielle pour l’Intelligence Artificielle de Défense Rapporteur, - M. Olivier SIMONIN Professeur Inria Chroma, Centre Inria de l'Université de Lyon Rapporteur, - M. Pierre-Yves OUDEYER Directeur de recherche Inria Flowers, Centre Inria de l'Université de Bordeaux Examinateur, - Mme Amy ZHANG Associate Professor The University of Texas at Austin Examinatrice, - M. Rémy PORTELAS Cadre scientifique Ubisoft Bordeaux Co-encadrant de thèse, - M. Ludovic DENOYER Imec AI-Labs Invité.

Thesis of the team SCOOL defended on 04/11/2026