Emotion Detection Through Facial Expression Recognition for a Social Robot

November 29, 2022

Having emotional awareness is critical to succeed when communicating with other people, even more so when the communication is between humans and robots. This work focuses on implementing Facial Expression Recognition on a social robot that operates in a hospital. Thus, several steps are taken: face detectors are compared and selected based on a created dataset; facial expression recognition datasets are generated based on available facial databases and prepro- cessed; and a set of facial expression recognition models are created and evaluated, making use of both conventional models (in particular Support Vector Machines and K Nearest Neighbors) and deep learn- ing models (popular conventional neural networks, such as Alexnet and VGG). The implementation of this project is done solely in Python, and the results of each model are presented and compared by an- alyzing commonly used Artificial Intelligence metrics, such as accuracy, mean squared error, F1 score, and confusion matrix, whilst keeping in mind the possible hardware limitations. In the end, the best model is chosen and implemented in a final program. Due to poor hardware and limited time, the final FER model performance is not as high as current state-of-the-art methods, however, since this project was built on top of several versatile Python scripts and programs that allow for more combinations and an increase of data in datasets, preprocessing and classifiers, better results can be achieved in future works.

Authors in the community:

• 

  F

  Francisco Manuel Raposo Esteves

  ist190075

Supervisors of this institution:

• 

  J

  João Fernando Cardoso Silva Sequeira

  ist13132

electrical-engineering-electronic-engineering-information-engineering - Electrical engineering, electronic engineering, information engineering

eng - English

Instituto Superior Técnico