Article In: cienciavitae

Group Synchrony for Emotion Recognition Using Physiological Signals

IEEE Transactions on Affective Computing

Bota, Patrícia; Zhang, Tianyi; Cesar, Pablo — 2023

Key information

Authors:

Bota, Patrícia (Patrícia Justo Bota); Zhang, Tianyi; El Ali, Abdallah; Fred, Ana; Hugo Plácido da Silva; Cesar, Pablo

Published in

October 1, 2023

Abstract

During group interactions, we react and modulate our emotions and behaviour to the group through phenomena including emotion contagion and physiological synchrony. Previous work on emotion recognition through video/image has shown that group context information improves the classification performance. However, when using physiological data, literature mostly focuses on intrapersonal models that leave-out group information, while interpersonal models are unexplored. This paper introduces a new interpersonal Weighted Group Synchrony approach, which relies on Electrodermal Activity (EDA) and Heart-Rate Variability (HRV). We perform an analysis of synchrony metrics applied across diverse data representations (EDA and HRV morphology and features, recurrence plot, spectrogram), to identify which metrics and modalities better characterise physiological synchrony for emotion recognition. We explored two datasets (AMIGOS and K-EmoCon), covering different group sizes (4 vs dyad) and group-based activities (video-watching vs conversation). The experimental results show that integrating group information improves arousal and valence classification, across all datasets, with the exception of K-EmoCon on valence. The proposed method was able to attain mean M-F1 of <inline-formula><tex-math notation="LaTeX">$\approxlt;/tex-math><alternatives><mml:math><mml:mo>&#x2248;</mml:mo></mml:math><inline-graphic xlink:href="bota-ieq1-3265433.gif"/></alternatives></inline-formula> 72.15&#x0025; arousal and 81.16&#x0025; valence for AMIGOS, and M-F1 of <inline-formula><tex-math notation="LaTeX">$\approxlt;/tex-math><alternatives><mml:math><mml:mo>&#x2248;</mml:mo></mml:math><inline-graphic xlink:href="bota-ieq2-3265433.gif"/></alternatives></inline-formula> 52.63&#x0025; arousal, 65.09&#x0025; valence for K-EmoCon, surpassing previous work results for K-EmoCon on arousal, and providing a new baseline on AMIGOS for long-videos.

Publication details

Authors in the community:

Publication version

AO - Author's Original

Title of the publication container

IEEE Transactions on Affective Computing

First page or article number

2614

Last page

2625

Volume

14

Issue

4

ISSN

2371-9850

Fields of Science and Technology (FOS)

other-engineering-and-technologies - Other engineering and technologies

Publication language (ISO code)

eng - English

Alternative identifier (URI)

http://dx.doi.org/10.1109/taffc.2023.3265433

Rights type:

Only metadata available