Visual intention of interaction for HRI using gaze, posture and gestures

Afonso Maria Gonçalves Soares

Key information

Authors:

Afonso Maria Gonçalves Soares (Afonso Maria Gonçalves Soares)

Supervisors:

Alexandre José Malheiro Bernardino (Alexandre José Malheiro Bernardino); José Alberto Rosado dos Santos Victor (José Santos Victor)

Published in

12/16/2019

Abstract

This master thesis proposes a tool to detect the intention of interaction of a human if he wants to interact with a robot, using an RGB camera as a sensor. This detection is divided into two sequential modules. The first is a potential intent detector, which aims to infer if a person is able to interact with the robot, that is, if it is a potential individual to interact with the robot. This detector consists of a multimodal system that uses the information of 3 modalities, the eye gaze, the head pose and the posture of the individual, being possible in the end to infer the potential intention to interact. The second module is a dynamic gesture detector that seeks to analyze the gestures performed by the human to initiate an interaction with the robot. The robot can conclude which gestures were made to perform an adequate response to what was performed. The proposed architecture allows gestures to be analyzed only if the human has a potential intention to interact. For each of the modalities, for their final combination and the processes associated with the gestures detector, classifiers were studied for each of these cases, to obtain the ones that lead to better performance. In the end, the whole tool was tested and evaluated to 31 subjects.