From Action Units to Deep Embeddings: A Comparative Study of Post-Stroke Classification under  Data Scarcity

Beatriz Bruto da Costa Gracias Rebelo

Key information

Authors:

Beatriz Bruto da Costa Gracias Rebelo (Beatriz Bruto da Costa Gracias Rebelo)

Supervisors:

Fabio Augusto Faria (Fabio Augusto Faria); Jacinto Carlos Marques Peixoto do Nascimento (Jacinto Carlos Marques Peixoto do Nascimento)

Published in

June 29, 2026

Abstract

Stroke is a leading cause of mortality and neurological disability. Current screening protocols like the FAST test rely on visual assessment and may miss subtle facial impairments from mild strokes. Automated facial analysis offers a promising alternative for computer-aided stroke assessment, though challenges like facial asymmetries, movement variability, and scarce annotated clinical datasets remain. This dissertation investigates different facial representations and temporal modeling strategies for automated post-stroke classification using facial videos from the Toronto NeuroFace dataset, which includes healthy controls and post-stroke patients performing standardized speech and non-speech tasks. We compared two facial representations: Action Unit (AU)-based features, which provide an interpretable description of facial muscle activity, and deep facial embeddings from a pretrained convolutional neural network. These were evaluated using Logistic Regression, Long Short-Term Memory (LSTM) networks, and attention-enhanced LSTM architectures. Principal Component Analysis (PCA) was also explored for dimensionality reduction in embedding-based representations. Results demonstrate that embedding-based temporal models consistently achieved strong performance, especially with PCA and attention mechanisms. The best embedding-based temporal model reached over 84% accuracy and 87% F1-score in global evaluation, significantly outperforming the baseline AU statistics model (60% accuracy, 72% F1-score). Results varied across tasks, while AU-based representations yielded competitive results for structured speech tasks, embedding-based approaches performed better across other tasks. These findings highlight the potential of deep facial representations and temporal attention mechanisms for future computer-aided post-stroke assessment systems.