Exploring Physiological Multimodality for Emotional Assessment

05/28/2019

Emotional responses combine subjective feeling and cognitive processes expressed by both motor and physiological manifestations. Many emotion recognition schemes have been proposed in the state-of-the-art. They generally differ in terms of the emotion elicitation methods, target emotional states to recognize, data sources or modalities, and classification techniques. In this work a multimodal approach based on biosignals is explored for emotion assessment during immersive video visualization, an elicitation method relatively unexplored within the related work. Data from Electrocardiography (ECG), Electrodermal Activity (EDA), Blood Volume Pulse (BVP) and Respiration sensors was collected, during a sequence of tasks comprising four calibrations and seven immersive videos, capable of eliciting different expected emotions. Participants reported their emotional state of the day (baseline), and provided self-assessment of the emotion experienced in each video through the Self-Assessment Manikin (SAM), in the valence-arousal space. Multiple physiological and statistical features extracted from the signals were used as inputs to an emotion recognition workflow, targeting both user-dependent and user-independent classification scenarios, with three and two classes per dimension, respectively. Support Vector Machines (SVM) were used, as it is considered one of the most promising classifiers in the field. The proposed approach led to accuracies of 51.07% for arousal and 67.68% for valence in the user-dependent approach, and 69.13% for arousal and 67.75% for valence in the user-independent approach, which are encouraging for further research with a larger training dataset and population.

Authors in the community:

• 

  J

  Joana Rosa Figueiredo Pinto

  ist178358

Supervisors of this institution:

• 

  A

  Ana Luísa Nobre Fred

  ist12170

industrial-biotechnology - Industrial Biotechnology

eng - English

Instituto Superior Técnico