Article In: cienciavitae
Group Synchrony for Emotion Recognition Using Physiological Signals
IEEE Transactions on Affective Computing
— 2023
Key information
Authors:
Published in
October 1, 2023
Abstract
During group interactions, we react and modulate our emotions and behaviour to the group through phenomena including emotion contagion and physiological synchrony. Previous work on emotion recognition through video/image has shown that group context information improves the classification performance. However, when using physiological data, literature mostly focuses on intrapersonal models that leave-out group information, while interpersonal models are unexplored. This paper introduces a new interpersonal Weighted Group Synchrony approach, which relies on Electrodermal Activity (EDA) and Heart-Rate Variability (HRV). We perform an analysis of synchrony metrics applied across diverse data representations (EDA and HRV morphology and features, recurrence plot, spectrogram), to identify which metrics and modalities better characterise physiological synchrony for emotion recognition. We explored two datasets (AMIGOS and K-EmoCon), covering different group sizes (4 vs dyad) and group-based activities (video-watching vs conversation). The experimental results show that integrating group information improves arousal and valence classification, across all datasets, with the exception of K-EmoCon on valence. The proposed method was able to attain mean M-F1 of <inline-formula><tex-math notation="LaTeX">$\approxlt;/tex-math><alternatives><mml:math><mml:mo>≈</mml:mo></mml:math><inline-graphic xlink:href="bota-ieq1-3265433.gif"/></alternatives></inline-formula> 72.15% arousal and 81.16% valence for AMIGOS, and M-F1 of <inline-formula><tex-math notation="LaTeX">$\approxlt;/tex-math><alternatives><mml:math><mml:mo>≈</mml:mo></mml:math><inline-graphic xlink:href="bota-ieq2-3265433.gif"/></alternatives></inline-formula> 52.63% arousal, 65.09% valence for K-EmoCon, surpassing previous work results for K-EmoCon on arousal, and providing a new baseline on AMIGOS for long-videos.
Publication details
Authors in the community:
Patrícia Justo Bota
ist194679
Publication version
AO - Author's Original
Title of the publication container
IEEE Transactions on Affective Computing
First page or article number
2614
Last page
2625
Volume
14
Issue
4
ISSN
2371-9850
Fields of Science and Technology (FOS)
other-engineering-and-technologies - Other engineering and technologies
Publication language (ISO code)
eng - English
Alternative identifier (URI)
http://dx.doi.org/10.1109/taffc.2023.3265433
Rights type:
Only metadata available