Thesis of Quentin De Smedt

Dynamic hand gesture recognition from traditional handcrafted to recent deep learning approaches

Hand gestures are the most natural and intuitive non-verbal communication medium while using a computer, and related research efforts have recently boosted interest. The area of hand gesture analysis covers hand pose estimation and gesture recognition. Hand pose estimation is considered to be more challenging than other human part estimation due to the small size of the hand, its greater complexity and its important self occlusions. Beside, the development of a precise hand gesture recognition system is also challenging due to high dissimilarities between gestures derived from ad-hoc, cultural and/or individual factors of users. First, we propose an original framework to represent hand gestures by using hand shape and motion descriptors computed on 3D hand skeletal features. Additionally, we create the Dynamic Hand Gesture dataset containing 14 gesture types. Evaluation results show the promising way of using hand skeletal data to perform hand gesture recognition. Then, we extend the study of hand gesture analysis to online recognition. Using a deep learning approach, we employ a transfer learning strategy to learn hand posture and shape features from depth image dataset originally created for hand pose estimation. Second, we model the temporal variations of the hand poses and its shapes using a recurrent deep learning technology. Finally, both information are merged to perform accurate prior detection and recognition of hand gestures. Experiments on two datasets demonstrate that the proposed approach is capable to detect an occurring gesture and to recognize its type far before its end.


- Directeur de thèse : Jean-Philippe VANDEBORRE - Co-encadrant : Hazem WANNOUS - Rapporteurs : Saida BOUAKAZ, Fabien MOUTARDE - Examinateurs : Laurent GRISONI, Francisco FLÓREZ-REVUELTA

Thesis of the team 3D SAM defended on 14/12/2017